Local Epochs Inefficiency Caused by Device Heterogeneity in Federated Learning

Federated learning is a new framework of machine learning, it trains models locally on multiple clients and then uploads local models to the server for model aggregation iteratively until the model converges. In most cases, the local epochs of all clients are set to the same value in federated learning. In practice, the clients are usually heterogeneous, which leads to the inconsistent training speed of clients. The faster clients will remain idle for a long time to wait for the slower clients, which prolongs the model training time. As the time cost of clients’ local training can reflect the clients’ training speed, and it can be used to guide the dynamic setting of local epochs, we propose a method based on deep learning to predict the training time of models on heterogeneous clients. First, a neural network is designed to extract the influence of different model features on training time. Second, we propose a dimensionality reduction rule to extract the key features which have a great impact on training time based on the influence of model features. Finally, we use the key features extracted by the dimensionality reduction rule to train the time prediction model. Our experiments show that, compared with the current prediction method, our method reduces 30% of model features and 25% of training data for the convolutional layer, 20% of model features and 20% of training data for the dense layer, while maintaining the same level of prediction error.

A/an and the: allomorphy or phonology?

English <em>a/an</em> presents a well-known paradox: it is restricted to a single morpheme, which suggests that it is a morphological phenomenon, yet it depends crucially on information about the <em>following</em> word, and thus cannot be characterized as a strictly word-internal process. Here I look at <em>a/an </em>alongside a strikingly similar but far less studied alternation: English definite article th/i/ vs. th/ə/. I argue that despite initial appearances, neither <em>a/an</em> nor <em>thi/t</em><em>hə</em> is determined by syllable well-formedness constraints. I show parallels between the two cases that can be explained under a phonological treatment featuring a vowel-reduction rule, which applies to both <em>thi</em> and the ‘strong’ <em>a/an</em> form /ej/.

The Rise and Fall of Swedish Wealth Taxation

We study the evolution of modern Swedish wealth taxation from its introduction in**1911 until it was abolished in 2007. The rules concerning valuation of assets, deductions/exemptions and tax schedules to characterize effective wealth tax schedules are described. These rules and schedules are used to calculate marginal and average wealth tax rates for three differently endowed owners of family firms and individual fortunes corresponding to a large, medium-sized and small firm. The overall trend in the direct wealth tax was rising until 1971 for owners of large and medium-sized firms and for individuals of equally-sized wealth consisting of non-corporate assets. Average direct wealth tax rates were low until 1934, except for 1913 when a progressive defense tax was levied. There were three major tax hikes: in 1934, when the wealth tax was more than doubled, in 1948 when tax rates were doubled again and in 1971 for owners of large firms and similarly sized non-corporate fortunes. Effective tax rates peaked in 1973 for owners of large firms and in 1983 for individuals with large non-corporate wealth. Reduction rules limited the wealth tax rates from 1934 for fortunes with high wealth/income ratios. The wealth tax on unlisted net business equity was abolished in 1991. Tax rates for wealthy individuals were decreased in 1991 and in 1992 and then remained at 0.5-1 percent through 2006, depending on whether the reduction rule was applicable. Tax rates for small-firm owners and small individual fortunes were substantially lower. Aggregate wealth tax revenues were rela-tively small; they never exceeded 0.4 percent of GDP in the postwar period and amounted to 0.16 percent of GDP in 2006.

Monadic translation of classical sequent calculus

We study monadic translations of the call-by-name (cbn) and call-by-value (cbv) fragments of the classical sequent calculus ${\overline{\lambda}\mu\tilde{\mu}}$ due to Curien and Herbelin, and give modular and syntactic proofs of strong normalisation. The target of the translations is a new meta-language for classical logic, named monadic λμ. This language is a monadic reworking of Parigot's λμ-calculus, where the monadic binding is confined to commands, thus integrating the monad with the classical features. Also, its μ-reduction rule is replaced by a rule expressing the interaction between monadic binding and μ-abstraction.Our monadic translations produce very tight simulations of the respective fragments of ${\overline{\lambda}\mu\tilde{\mu}}$ within monadic λμ, with reduction steps of ${\overline{\lambda}\mu\tilde{\mu}}$ being translated in a 1–1 fashion, except for β steps, which require two steps. The monad of monadic λμ can be instantiated to the continuations monad so as to ensure strict simulation of monadic λμ within simply typed λ-calculus with β- and η-reduction. Through strict simulation, the strong normalisation of simply typed λ-calculus is inherited by monadic λμ, and then by cbn and cbv ${\overline{\lambda}\mu\tilde{\mu}}$, thus reproving strong normalisation in an elementary syntactical way for these fragments of ${\overline{\lambda}\mu\tilde{\mu}}$, and establishing it for our new calculus. These results extend to second-order logic, with polymorphic λ-calculus as the target, giving new strong normalisation results for classical second-order logic in sequent calculus style.CPS translations of cbn and cbv ${\overline{\lambda}\mu\tilde{\mu}}$ with the strict simulation property are obtained by composing our monadic translations with the continuations-monad instantiation. In an appendix to the paper, we investigate several refinements of the continuations-monad instantiation in order to obtain in a modular way improvements of the CPS translations enjoying extra properties like simulation by cbv β-reduction or reduction of administrative redexes at compile time.