Sign-rank Can Increase under Intersection

The communication class UPP cc is a communication analog of the Turing Machine complexity class PP . It is characterized by a matrix-analytic complexity measure called sign-rank (also called dimension complexity), and is essentially the most powerful communication class against which we know how to prove lower bounds. For a communication problem f , let f ∧ f denote the function that evaluates f on two disjoint inputs and outputs the AND of the results. We exhibit a communication problem f with UPP cc ( f ) = O (log n ), and UPP cc ( f ∧ f ) = Θ (log 2 n ). This is the first result showing that UPP communication complexity can increase by more than a constant factor under intersection. We view this as a first step toward showing that UPP cc , the class of problems with polylogarithmic-cost UPP communication protocols, is not closed under intersection. Our result shows that the function class consisting of intersections of two majorities on n bits has dimension complexity n Omega Ω(log n ) . This matches an upper bound of (Klivans, O’Donnell, and Servedio, FOCS 2002), who used it to give a quasipolynomial time algorithm for PAC learning intersections of polylogarithmically many majorities. Hence, fundamentally new techniques will be needed to learn this class of functions in polynomial time.

Towards an Adaptive Educational Course on the Mathematical Foundations of Machine Learning

Today the development of information technology is closely related to the creation and application of machine learning and data analysis methods. In this regard, the need for training specialists in this area is growing. Very often, the study of machine learning methods is combined with the study of a certain programming language and the tools of its specialized library. This approach is undoubtedly justified, because it provides the possibility of accelerated application of the knowledge gained in practice. At the same time, it should be noted that with this approach, it is rather not machine learning methods that are studied, but a certain set of methodological techniques for using the tools of the specialized library. The presented work is devoted to the experience of creating an adaptive educational course on the mathematical foundations of machine learning. This course is aimed at undergraduate and graduate students of mathematical specialties. It is divided into core and variable parts. The obligatory core part is built around the PAC learning model and the binary classification problem. Within the variable part, issues of the weak learning model and the boosting methods are considered. Also a methodology of changing the variable part of the course is discussed.

On the Complexity of Learning a Class Ratio from Unlabeled Data

In the problem of learning a class ratio from unlabeled data, which we call CR learning, the training data is unlabeled, and only the ratios, or proportions, of examples receiving each label are given. The goal is to learn a hypothesis that predicts the proportions of labels on the distribution underlying the sample. This model of learning is applicable to a wide variety of settings, including predicting the number of votes for candidates in political elections from polls. In this paper, we formally define this class and resolve foundational questions regarding the computational complexity of CR learning and characterize its relationship to PAC learning. Among our results, we show, perhaps surprisingly, that for finite VC classes what can be efficiently CR learned is a strict subset of what can be learned efficiently in PAC, under standard complexity assumptions. We also show that there exist classes of functions whose CR learnability is independent of ZFC, the standard set theoretic axioms. This implies that CR learning cannot be easily characterized (like PAC by VC dimension).

Sequential Mode Estimation with Oracle Queries

We consider the problem of adaptively PAC-learning a probability distribution 𝒫's mode by querying an oracle for information about a sequence of i.i.d. samples X1, X2, … generated from 𝒫. We consider two different query models: (a) each query is an index i for which the oracle reveals the value of the sample Xi, (b) each query is comprised of two indices i and j for which the oracle reveals if the samples Xi and Xj are the same or not. For these query models, we give sequential mode-estimation algorithms which, at each time t, either make a query to the corresponding oracle based on past observations, or decide to stop and output an estimate for the distribution's mode, required to be correct with a specified confidence. We analyze the query complexity of these algorithms for any underlying distribution 𝒫, and derive corresponding lower bounds on the optimal query complexity under the two querying models.