Search for contacts, projects,
courses and publications

New Econometric Methods for Big Data



Gagliardini P.


Carlini F.


Ma H.



In recent years, empirical research in Economics and Finance has benefited from the growing availability of large-dimensional datasets, often referred to as "big data". While such large datasets pose new methodological challenges in terms of modeling and statistical analysis, they also offer valuable opportunities to address new relevant questions. The aim of this research project is to develop and study new econometric methods for the analysis of large-dimensional datasets, and demonstrate their usefulness in a variety of applications in Economics and Finance, in particular for asset pricing and systematic risks analysis. The project focuses on panel data, which are datasets of doubly-indexed observations. Panel data are obtained when the behaviour of a set of individuals is repeatedly observed at several time points. The "behaviour" of "individuals" can correspond to economic decisions of consumers, firms and countries, or returns of financial assets, forecasts of financial analysts, the output of different macro-economic sectors, etc. In such cases, the observations are indexed by individual and time. Panel data can also consist of the observations on the intensity of interactions between pairs of individuals i and j, say, like commercial trade between countries i and j, or the amount of purchase of good i by costumer j. We are especially interested in settings where the panel dimensions, i.e. the number of individuals and number of time dates, are large. For instance, this is the case for datasets reporting the histories of returns and financial characteristics for several thousands of individual companies listed in a large stock exchange, or for internal datasets containing information on retail credit, credit cards, e-commerce for millions of clients. We develop structural modeling and statistical inference methodologies to extract time series and cross-sectional information from such large panel datasets. The focus of our analysis is on understanding the interdependencies between individual behaviours, or individual risks. We adopt the paradigm offered by factor models, which explain interdependencies among individual histories in terms of their exposure to common sources of uncertainty, called systematic factors. For instance, which systematic risk factors explain the co-movements across the returns of individual stocks? Can we estimate the time-varying compensation asked by investors for bearing systematic risk? Can we disentangle direct causality effects, i.e. contagion, from exposure to common sources of dependence? This project advances the econometric analysis of large-dimensional factor models along several dimensions. In the first subproject, we study models that allow for unobservable factors and time-varying factor loadings. In a conditional factor model, the loadings change as an effect of the changing economic environment. The goal is to arrive at the concept of a conditional factor structure without identifying the factors with economic variables specified a priori. In the second subproject, we study the possibility to disentangle contagion effects from latent common factors in large-dimensional dynamic models. We consider nonlinear latent factor models in the third subproject. Nonlinear specifications are necessary when the measurement is a qualitative variable, or has a limited support. In the fourth subproject, we study factor models with endogenous partial observability. This step addresses the potential endogeneity beyond the unbalanced nature of large panels of individual histories. A common treat in the econometric analysis proposed in this project is the use of cross-sectional averaging methods and instrumental variables. The intuition is that the cross-sectional averages of well-chosen functions of the data span the factor space. This approach allows us to obtain estimators with low numerical complexity and tractable large sample properties. The project team is composed by the PI, one post-doc and one PhD student. The scientific results will be the content of five research papers to be submitted to international top journals in Economics and top field journals in Econometrics. The research themes of this project are important both for their topical relevance in the academic community and for the implications in economic policy and financial regulation. After the recent financial crisis, systematic risks and contagion effects have become central topics of concern for supervisory authorities and are at the core of the modern regulations.

Additional information

Start date
End date
42 Months
Funding sources
Swiss National Science Foundation / Project Funding / Humanities and social sciences (Division I)