<?xml version="1.0" encoding="UTF-8"?>
 <rdf:RDF xmlns="http://purl.org/rss/1.0/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://web.resource.org/cc/" xmlns:syn="http://purl.org/rss/1.0/modules/syndication/" xmlns:admin="http://webns.net/mvcb/">
  <channel rdf:about="http://pinboard.in">
    <title>Pinboard (cshalizi)</title>
    <link>https://pinboard.in/u:cshalizi/public/</link>
    <description>recent bookmarks from cshalizi</description>
    <items>
      <rdf:Seq>	<rdf:li rdf:resource="https://openreview.net/forum?id=ib0aV2hphN"/>
	<rdf:li rdf:resource="https://onlinelibrary.wiley.com/doi/full/10.1111/jtsa.70025?campaign=wolearlyview"/>
	<rdf:li rdf:resource="https://projecteuclid.org/journals/electronic-journal-of-statistics/volume-19/issue-2/On-the-rate-of-convergence-of-an-over-parametrized-deep/10.1214/25-EJS2444.full"/>
	<rdf:li rdf:resource="https://projecteuclid.org/journals/annals-of-statistics/volume-48/issue-4/Nonparametric-regression-using-deep-neural-networks-with-ReLU-activation-function/10.1214/19-AOS1875.full"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2203.06469"/>
	<rdf:li rdf:resource="https://journal.r-project.org/archive/2016/RJ-2016-052/RJ-2016-052.pdf"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2502.04309"/>
	<rdf:li rdf:resource="https://www.nber.org/papers/w33349"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2305.04116"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2402.16422"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2309.17016"/>
	<rdf:li rdf:resource="https://projecteuclid.org/journals/annals-of-statistics/volume-21/issue-4/Comparing-Nonparametric-Versus-Parametric-Regression-Fits/10.1214/aos/1176349403.full"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2302.06005"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2112.03626"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2008.02915"/>
	<rdf:li rdf:resource="https://www.nber.org/papers/w29709"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2107.07257"/>
	<rdf:li rdf:resource="https://projecteuclid.org/journals/electronic-journal-of-statistics/volume-15/issue-2/Spectrally-truncated-kernel-ridge-regression-and-its-free-lunch/10.1214/21-EJS1873.full"/>
	<rdf:li rdf:resource="https://jmlr.org/papers/v22/19-892.html"/>
	<rdf:li rdf:resource="https://jmlr.org/papers/v22/20-1369.html"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2106.01529"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2105.15197"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2105.14075"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1904.00989"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2105.04754"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2012.13106"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2104.10601"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2010.04855"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2104.09935"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1406.5362"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2004.14497"/>
	<rdf:li rdf:resource="https://www.tandfonline.com/doi/full/10.1080/01621459.2021.1895175"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1802.08667"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1809.05224"/>
	<rdf:li rdf:resource="https://onlinelibrary.wiley.com/doi/10.1111/sjos.12529"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2101.10255"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1806.05161"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1806.09471"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2101.04783"/>
	<rdf:li rdf:resource="https://www.pnas.org/content/113/38/10530"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2101.01535"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2012.14482"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2012.14563"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2006.11937"/>
	<rdf:li rdf:resource="https://projecteuclid.org/euclid.ss/1608541216"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2007.04286"/>
	<rdf:li rdf:resource="https://academic.oup.com/biomet/article-abstract/107/4/983/5857283?redirectedFrom=fulltext"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2005.06371"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2012.08444"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2012.03376"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2011.12215"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1712.00038"/>
	<rdf:li rdf:resource="https://www.annualreviews.org/doi/abs/10.1146/annurev-control-053018-023744"/>
	<rdf:li rdf:resource="https://projecteuclid.org/euclid.aos/1600480929"/>
	<rdf:li rdf:resource="https://projecteuclid.org/euclid.aos/1594972839"/>
	<rdf:li rdf:resource="https://projecteuclid.org/euclid.aos/1594972822"/>
	<rdf:li rdf:resource="https://projecteuclid.org/euclid.aos/1594972820"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/2011.07275"/>
	<rdf:li rdf:resource="https://projecteuclid.org/euclid.ejs/1597975224"/>
	<rdf:li rdf:resource="https://projecteuclid.org/euclid.ejs/1576573369"/>
	<rdf:li rdf:resource="https://ieeexplore.ieee.org/document/9136791"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1910.00943"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1908.03606"/>
	<rdf:li rdf:resource="https://projecteuclid.org/euclid.ejs/1574240425"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1807.05748"/>
	<rdf:li rdf:resource="https://projecteuclid.org/euclid.aos/1572487393"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1907.02306"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1603.07632"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1910.08477"/>
	<rdf:li rdf:resource="https://arxiv.org/abs/1910.06028"/>
      </rdf:Seq>
    </items>
  </channel><item rdf:about="https://openreview.net/forum?id=ib0aV2hphN">
    <title>High-Order Flow Matching: Unified Framework and Sharp Statistical Rates | OpenReview</title>
    <dc:date>2026-01-30T12:12:17+00:00</dc:date>
    <link>https://openreview.net/forum?id=ib0aV2hphN</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Flow matching is an emerging generative modeling framework that learns continuous-time dynamics to map noise into data. To enhance expressiveness and sampling efficiency, recent works have explored incorporating high-order trajectory information. Despite the empirical success, a holistic theoretical foundation is still lacking. We present a unified framework for standard and high-order flow matching that incorporates trajectory derivatives up to an arbitrary order $K$. Our key innovation is establishing the marginalization technique that converts the intractable $K$-order loss into a simple conditional regression with exact gradients and identifying the consistency constraint. We establish sharp statistical rates of the $K$-order flow matching implemented with transformer networks. With 
$n$ samples, flow matching estimates nonparametric distributions at a rate , matching minimax lower bounds up to logarithmic factors."]]></description>
<dc:subject>to:NB density_estimation neural_networks nonparametrics liu.han</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:5033f0d08de2/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:density_estimation"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:neural_networks"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:liu.han"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://onlinelibrary.wiley.com/doi/full/10.1111/jtsa.70025?campaign=wolearlyview">
    <title>Density‐Valued ARMA Models by Spline Mixtures - Matsuda - Journal of Time Series Analysis - Wiley Online Library</title>
    <dc:date>2025-10-25T20:02:50+00:00</dc:date>
    <link>https://onlinelibrary.wiley.com/doi/full/10.1111/jtsa.70025?campaign=wolearlyview</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["This paper proposes a novel framework for modeling time series of probability density functions by extending autoregressive moving average (ARMA) models to density-valued data. The method is based on a transformation approach, wherein each density function on a compact domain $[0,1]^d$ is approximated by a B-spline mixture representation. Through generalized logit and softmax mappings, the space of density functions is transformed into an unconstrained Euclidean space, enabling the application of classical time series techniques. We define ARMA-type dynamics in the transformed space. Estimation is carried out via least squares for density-valued AR models and Whittle likelihood for ARMA models, with asymptotic normality derived under the joint divergence of the time horizon and basis dimension. The proposed methodology is applied to spatiotemporal human population data in Tokyo, where meaningful temporal structures in the distributional dynamics are successfully captured."]]></description>
<dc:subject>to:NB splines time_series nonparametrics to_read</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:2c29c195dd77/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:splines"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:time_series"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/journals/electronic-journal-of-statistics/volume-19/issue-2/On-the-rate-of-convergence-of-an-over-parametrized-deep/10.1214/25-EJS2444.full">
    <title>On the rate of convergence of an over-parametrized deep neural network regression estimate with ReLU activation function learned by gradient descent</title>
    <dc:date>2025-10-24T19:40:42+00:00</dc:date>
    <link>https://projecteuclid.org/journals/electronic-journal-of-statistics/volume-19/issue-2/On-the-rate-of-convergence-of-an-over-parametrized-deep/10.1214/25-EJS2444.full</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Estimation of a regression function from independent and identically distributed random variables is considered. The $L_2$ error with integration with respect to the design measure is used as an error criterion. Over-parametrized deep neural network estimates with ReLU activation function are defined where all the weights are learned by the gradient descent. It is shown that the expected $L_2$ error of the estimates converges to zero with rate
\[
n^{-\frac{p}{2p+d}
\]
(up to some logarithmic factor) in case that the regression function is p-times continuously differentiable. In case that the regression function satisfies the assumption of a p times continuously differentiable interaction model, i.e., in case that it is equal to a finite sum of functions where each function in the sum is a p-times continuously differentiable function applied to only $d^*$ of the d components of its input, we show that our estimate achieves the above rate of convergence with d replaced by $d^*$. The finite sample performance of the proposed estimate has been illustrated by simulations."]]></description>
<dc:subject>to:NB regression neural_networks learning_theory nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:deab690bc7f0/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:neural_networks"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/journals/annals-of-statistics/volume-48/issue-4/Nonparametric-regression-using-deep-neural-networks-with-ReLU-activation-function/10.1214/19-AOS1875.full">
    <title>Nonparametric regression using deep neural networks with ReLU activation function</title>
    <dc:date>2025-08-07T14:59:12+00:00</dc:date>
    <link>https://projecteuclid.org/journals/annals-of-statistics/volume-48/issue-4/Nonparametric-regression-using-deep-neural-networks-with-ReLU-activation-function/10.1214/19-AOS1875.full</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to $\log{n}$-factors) under a general composition assumption on the regression function. The framework includes many well-studied structural constraints such as (generalized) additive models. While there is a lot of flexibility in the network architecture, the tuning parameter is the sparsity of the network. Specifically, we consider large networks with number of potential network parameters exceeding the sample size. The analysis gives some insights into why multilayer feedforward neural networks perform well in practice. Interestingly, for ReLU activation function the depth (number of layers) of the neural network architectures plays an important role, and our theory suggests that for nonparametric regression, scaling the network depth with the sample size is natural. It is also shown that under the composition assumption wavelet estimators can only achieve suboptimal rates."]]></description>
<dc:subject>to:NB neural_networks regression nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:bec55e957341/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:neural_networks"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2203.06469">
    <title>[2203.06469] Semiparametric doubly robust targeted double machine learning: a review</title>
    <dc:date>2025-04-28T01:51:30+00:00</dc:date>
    <link>https://arxiv.org/abs/2203.06469</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["In this review we cover the basics of efficient nonparametric parameter estimation (also called functional estimation), with a focus on parameters that arise in causal inference problems. We review both efficiency bounds (i.e., what is the best possible performance for estimating a given parameter?) and the analysis of particular estimators (i.e., what is this estimator's error, and does it attain the efficiency bound?) under weak assumptions. We emphasize minimax-style efficiency bounds, worked examples, and practical shortcuts for easing derivations. We gloss over most technical details, in the interest of highlighting important concepts and providing intuition for main ideas."

--- I need to revise the causal inference chapters (and problem sets?) to at least mention this stuff.

--- ETA after reading:
1. I'm relieved that the title is a joke.  (I kind of suspected, knowing Ed, but the first footnote is still re-assuring.)
2. The notation in this area sucks, _I_ have trouble keeping track of $\psi$ vs $\varphi$ (and I think I saw a $\phi$ which may have just been a typo?) vs...
3. I'm a little surprised that EHK doesn't get into issues about total estimation error, as opposed to just bias-correction followed by confidence intervals.
4. As for incorporating this into ADAfaEPoV, there's _probably_ a way, but it'll need a lot of writing on my part.  I _could_ just say "here're the bias correction terms for these cases", but that seems wrong, contrary to the spirit of the book.  OTOH going that deep into influence functions, well...]]></description>
<dc:subject>causal_inference nonparametrics kith_and_kin kennedy.edward_h. re:ADAfaEPoV have_read in_NB</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:0dbbe6f7f177/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_inference"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kith_and_kin"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kennedy.edward_h."/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:re:ADAfaEPoV"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:have_read"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://journal.r-project.org/archive/2016/RJ-2016-052/RJ-2016-052.pdf">
    <title>quantreg.nonpar: An R Package for Performing Nonparametric Series Quantile Regression</title>
    <dc:date>2025-03-02T14:59:23+00:00</dc:date>
    <link>https://journal.r-project.org/archive/2016/RJ-2016-052/RJ-2016-052.pdf</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["by Michael Lipsitz, Alexandre Belloni, Victor Chernozhukov, and Iván Fernández-Val
"Abstract The R package quantreg.nonpar implements nonparametric quantile regression methods to estimate and make inference on partially linear quantile models. quantreg.nonpar obtains point estimates of the conditional quantile function and its derivatives based on series approximations to the nonparametric part of the model. It also provides pointwise and uniform confidence intervals over a region of covariate values and/or quantile indices for the same functions using analytical and resampling methods. This paper serves as an introduction to the package and displays basic functionality of the functions contained within."]]></description>
<dc:subject>to:NB nonparametrics quantile_regression re:codename:catherine_wheel to_read</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:18bdd27ba880/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:quantile_regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:re:codename:catherine_wheel"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2502.04309">
    <title>[2502.04309] Targeted Learning for Data Fairness</title>
    <dc:date>2025-02-11T13:39:36+00:00</dc:date>
    <link>https://arxiv.org/abs/2502.04309</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Data and algorithms have the potential to produce and perpetuate discrimination and disparate treatment. As such, significant effort has been invested in developing approaches to defining, detecting, and eliminating unfair outcomes in algorithms. In this paper, we focus on performing statistical inference for fairness. Prior work in fairness inference has largely focused on inferring the fairness properties of a given predictive algorithm. Here, we expand fairness inference by evaluating fairness in the data generating process itself, referred to here as data fairness. We perform inference on data fairness using targeted learning, a flexible framework for nonparametric inference. We derive estimators demographic parity, equal opportunity, and conditional mutual information. Additionally, we find that our estimators for probabilistic metrics exploit double robustness. To validate our approach, we perform several simulations and apply our estimators to real data."

--- I dunno about "is the conditional mutual information positive?" being a _fairness_ criterion.  But if you are going to go that way, [http://arxiv.org/abs/1412.464]]]></description>
<dc:subject>to:NB algorithmic_fairness hooker.giles entropy_estimation nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:03e80943f40f/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:algorithmic_fairness"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:hooker.giles"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:entropy_estimation"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://www.nber.org/papers/w33349">
    <title>Accounting for Individual-Specific Heterogeneity in Intergenerational Income Mobility | NBER</title>
    <dc:date>2025-01-21T02:07:46+00:00</dc:date>
    <link>https://www.nber.org/papers/w33349</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["This paper proposes a fully nonparametric model to investigate the dynamics of intergenerational income mobility for discrete outcomes. In our model, an individual’s income class probabilities depend on parental income in a manner that accommodates nonlinearities and interactions among various individual and parental characteristics, including race, education, and parental age at childbearing. Consequently, we offer a generalization of Markov chain mobility models. We employ kernel techniques from machine learning and further regularization for estimating this highly flexible model. Utilizing data from the Panel Study of Income Dynamics (PSID), we find that race and parental education interact with parental income in children’s economic prospects in ways that can create bottlenecks in mobility."]]></description>
<dc:subject>to:NB economics to_read nonparametrics transmission_of_inequality to_teach:statistics_of_inequality_and_discrimination durlauf.steven_n.</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:e3273025483b/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:economics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:transmission_of_inequality"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_teach:statistics_of_inequality_and_discrimination"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:durlauf.steven_n."/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2305.04116">
    <title>[2305.04116] The Fundamental Limits of Structure-Agnostic Functional Estimation</title>
    <dc:date>2024-12-11T19:37:38+00:00</dc:date>
    <link>https://arxiv.org/abs/2305.04116</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Many recent developments in causal inference, and functional estimation problems more generally, have been motivated by the fact that classical one-step (first-order) debiasing methods, or their more recent sample-split double machine-learning avatars, can outperform plugin estimators under surprisingly weak conditions. These first-order corrections improve on plugin estimators in a black-box fashion, and consequently are often used in conjunction with powerful off-the-shelf estimation methods. These first-order methods are however provably suboptimal in a minimax sense for functional estimation when the nuisance functions live in Holder-type function spaces. This suboptimality of first-order debiasing has motivated the development of "higher-order" debiasing methods. The resulting estimators are, in some cases, provably optimal over Holder-type spaces, but both the estimators which are minimax-optimal and their analyses are crucially tied to properties of the underlying function space.
"In this paper we investigate the fundamental limits of structure-agnostic functional estimation, where relatively weak conditions are placed on the underlying nuisance functions. We show that there is a strong sense in which existing first-order methods are optimal. We achieve this goal by providing a formalization of the problem of functional estimation with black-box nuisance function estimates, and deriving minimax lower bounds for this problem. Our results highlight some clear tradeoffs in functional estimation -- if we wish to remain agnostic to the underlying nuisance function spaces, impose only high-level rate conditions, and maintain compatibility with black-box nuisance estimators then first-order methods are optimal. When we have an understanding of the structure of the underlying nuisance functions then carefully constructed higher-order estimators can outperform first-order estimators."]]></description>
<dc:subject>to:NB to_read statistics nonparametrics entropy_estimation kith_and_kin kennedy.edward_h. wasserman.larry balakrishnan.sivaraman causal_inference</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:0ee8eb04a835/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:entropy_estimation"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kith_and_kin"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kennedy.edward_h."/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:wasserman.larry"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:balakrishnan.sivaraman"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_inference"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2402.16422">
    <title>[2402.16422] Bayesian nonparametric statistics, St-Flour lecture notes</title>
    <dc:date>2024-02-27T20:00:22+00:00</dc:date>
    <link>https://arxiv.org/abs/2402.16422</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["These are lecture notes of the 51st Saint-Flour summer school, July 2023, on the topic of Bayesian nonparametric statistics"
--- 186pp]]></description>
<dc:subject>to:NB nonparametrics bayesianism</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:88ac8b8a01b7/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:bayesianism"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2309.17016">
    <title>[2309.17016] Efficient Agnostic Learning with Average Smoothness</title>
    <dc:date>2023-12-08T17:15:57+00:00</dc:date>
    <link>https://arxiv.org/abs/2309.17016</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We study distribution-free nonparametric regression following a notion of average smoothness initiated by Ashlagi et al. (2021), which measures the "effective" smoothness of a function with respect to an arbitrary unknown underlying distribution. While the recent work of Hanneke et al. (2023) established tight uniform convergence bounds for average-smooth functions in the realizable case and provided a computationally efficient realizable learning algorithm, both of these results currently lack analogs in the general agnostic (i.e. noisy) case.
"In this work, we fully close these gaps. First, we provide a distribution-free uniform convergence bound for average-smoothness classes in the agnostic setting. Second, we match the derived sample complexity with a computationally efficient agnostic learning algorithm. Our results, which are stated in terms of the intrinsic geometry of the data and hold over any totally bounded metric space, show that the guarantees recently obtained for realizable learning of average-smooth functions transfer to the agnostic setting. At the heart of our proof, we establish the uniform convergence rate of a function class in terms of its bracketing entropy, which may be of independent interest."]]></description>
<dc:subject>in_NB nonparametrics learning_theory empirical_processes kith_and_kin kontorovich.aryeh hanneke.steve</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:372bb023a730/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:empirical_processes"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kith_and_kin"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kontorovich.aryeh"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:hanneke.steve"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/journals/annals-of-statistics/volume-21/issue-4/Comparing-Nonparametric-Versus-Parametric-Regression-Fits/10.1214/aos/1176349403.full">
    <title>Comparing Nonparametric Versus Parametric Regression Fits</title>
    <dc:date>2023-05-01T19:55:30+00:00</dc:date>
    <link>https://projecteuclid.org/journals/annals-of-statistics/volume-21/issue-4/Comparing-Nonparametric-Versus-Parametric-Regression-Fits/10.1214/aos/1176349403.full</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["In general, there will be visible differences between a parametric and a nonparametric curve estimate. It is therefore quite natural to compare these in order to decide whether the parametric model could be justified. An asymptotic quantification is the distribution of the integrated squared difference between these curves. We show that the standard way of bootstrapping this statistic fails. We use and analyse a different form of bootstrapping for this task. We call this method the wild bootstrap and apply it to fitting Engel curves in expenditure data analysis."]]></description>
<dc:subject>regression goodness-of-fit model_checking nonparametrics bootstrap hardle.wolfgang re:ADAfaEPoV cleaning_out_the_filing_cabinet_for_the_first_time_since_2005 in_NB have_read</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:7023957251ed/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:goodness-of-fit"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:model_checking"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:bootstrap"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:hardle.wolfgang"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:re:ADAfaEPoV"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:cleaning_out_the_filing_cabinet_for_the_first_time_since_2005"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:have_read"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2302.06005">
    <title>[2302.06005] Near-optimal learning with average Hölder smoothness</title>
    <dc:date>2023-02-24T03:27:08+00:00</dc:date>
    <link>https://arxiv.org/abs/2302.06005</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We generalize the notion of average Lipschitz smoothness proposed by Ashlagi et al. (COLT 2021) by extending it to Hölder smoothness. This measure of the ``effective smoothness'' of a function is sensitive to the underlying distribution and can be dramatically smaller than its classic ``worst-case'' Hölder constant. We prove nearly tight upper and lower risk bounds in terms of the average Hölder smoothness, establishing the minimax rate in the realizable regression setting up to log factors; this was not previously known even in the special case of average Lipschitz smoothness. From an algorithmic perspective, since our notion of average smoothness is defined with respect to the unknown sampling distribution, the learner does not have an explicit representation of the function class, hence is unable to execute ERM. Nevertheless, we provide a learning algorithm that achieves the (nearly) optimal learning rate. Our results hold in any totally bounded metric space, and are stated in terms of its intrinsic geometry. Overall, our results show that the classic worst-case notion of Hölder smoothness can be essentially replaced by its average, yielding considerably sharper guarantees."]]></description>
<dc:subject>to:NB learning_theory nonparametrics kontorovich.aryeh hanneke.steve</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:3454a701b130/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kontorovich.aryeh"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:hanneke.steve"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2112.03626">
    <title>[2112.03626] Phase transitions in nonparametric regressions: a curse of exploiting higher degree smoothness assumptions in finite samples</title>
    <dc:date>2022-07-25T17:15:55+00:00</dc:date>
    <link>https://arxiv.org/abs/2112.03626</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["When the regression function belongs to a smooth class consisting of univariate functions with derivatives up to the (γ+1)th order bounded in absolute values for a finite γ, it is generally viewed that exploiting higher degree smoothness assumptions helps reduce the estimation error. This paper shows that the minimax optimal mean integrated squared error (MISE) increases in γ when the sample size n is small relative to the order of (γ+1)2γ+3, and decreases in γ when n is large relative to the order of (γ+1)2γ+3. In particular, this phase transition property is shown to be achieved by common nonparametric procedures. Consider γ1 and γ2 such that γ1<γ2, where the (γ2+1)th degree smoothness class is a subset of the (γ1+1)th degree class. What is surprising about our results is that they imply, if n is small relative to the order of (γ1+1)2γ1+3, the optimal rate achieved by the estimator constrained to be in the smoother class (to exploit the (γ2+1)th degree smoothness) is slower. In data sets with fewer than hundreds-of-thousands observations, our results suggest that one should not exploit beyond the third or fourth degree of smoothness. To some extent, our results provide a theoretical basis for the widely adopted practical recommendations given by Gelman and Imbens (2019).
"The building blocks of our minimax optimality results are a set of metric entropy bounds we develop in this paper for smooth function classes. Some of our bounds are original, and some of them improve and/or generalize the ones in the literature."

--- This is really surprising to me, so I ought to see what makes it work.  (On the plus side, if right, it makes me feel better about not teaching The Kids about higher-order smoothness assumptions!)]]></description>
<dc:subject>in_NB regression nonparametrics minimax learning_theory statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:e1ba3a57ef77/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:minimax"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2008.02915">
    <title>[2008.02915] Kernel Ordinary Differential Equations</title>
    <dc:date>2022-02-15T14:55:41+00:00</dc:date>
    <link>https://arxiv.org/abs/2008.02915</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Ordinary differential equation (ODE) is widely used in modeling biological and physical processes in science. In this article, we propose a new reproducing kernel-based approach for estimation and inference of ODE given noisy observations. We do not assume the functional forms in ODE to be known, or restrict them to be linear or additive, and we allow pairwise interactions. We perform sparse estimation to select individual functionals, and construct confidence intervals for the estimated signal trajectories. We establish the estimation optimality and selection consistency of kernel ODE under both the low-dimensional and high-dimensional settings, where the number of unknown functionals can be smaller or larger than the sample size. Our proposal builds upon the smoothing spline analysis of variance (SS-ANOVA) framework, but tackles several important problems that are not yet fully addressed, and thus extends the scope of existing SS-ANOVA too. We demonstrate the efficacy of our method through numerous ODE examples."]]></description>
<dc:subject>equations_of_motion_from_a_time_series dynamical_systems kernel_methods statistics nonparametrics splines in_NB have_skimmed</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:8188454a9772/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:equations_of_motion_from_a_time_series"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:dynamical_systems"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kernel_methods"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:splines"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:have_skimmed"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://www.nber.org/papers/w29709">
    <title>When is TSLS Actually LATE? | NBER</title>
    <dc:date>2022-02-02T23:00:41+00:00</dc:date>
    <link>https://www.nber.org/papers/w29709</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Linear instrumental variable estimators, such as two-stage least squares (TSLS), are commonly interpreted as estimating positively weighted averages of causal effects, referred to as local average treatment effects (LATEs). We examine whether the LATE interpretation actually applies to the types of TSLS specifications that are used in practice. We show that if the specification includes covariates, which most empirical work does, then the LATE interpretation does not apply in general. Instead, the TSLS estimator will in general reflect treatment effects for both compliers and always/never-takers, and some of the treatment effects for the always/never-takers will necessarily be negatively weighted. We show that the only specifications that have a LATE interpretation are "saturated" specifications that control for covariates nonparametrically, implying that such specifications are both sufficient and necessary for TSLS to have a LATE interpretation, at least without additional parametric assumptions. This result is concerning because, as we document, empirical researchers almost never control for covariates nonparametrically, and rarely discuss or justify parametric specifications of covariates. We develop a decomposition that quantifies the extent to which the usual LATE interpretation fails. We apply the decomposition to four empirical analyses and find strong evidence that the LATE interpretation of TSLS is far from accurate for the types of specifications actually used in practice."]]></description>
<dc:subject>instrumental_variables causal_inference nonparametrics re:ADAfaEPoV in_NB</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:4f733f8f89d2/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:instrumental_variables"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_inference"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:re:ADAfaEPoV"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2107.07257">
    <title>[2107.07257] Nonparametric, tuning-free estimation of S-shaped functions</title>
    <dc:date>2021-08-13T16:14:59+00:00</dc:date>
    <link>https://arxiv.org/abs/2107.07257</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We consider the nonparametric estimation of an S-shaped regression function. The least squares estimator provides a very natural, tuning-free approach, but results in a non-convex optimisation problem, since the inflection point is unknown. We show that the estimator may nevertheless be regarded as a projection onto a finite union of convex cones, which allows us to propose a mixed primal-dual bases algorithm for its efficient, sequential computation. After developing a projection framework that demonstrates the consistency and robustness to misspecification of the estimator, our main theoretical results provide sharp oracle inequalities that yield worst-case and adaptive risk bounds for the estimation of the regression function, as well as a rate of convergence for the estimation of the inflection point. These results reveal not only that the estimator achieves the minimax optimal rate of convergence for both the estimation of the regression function and its inflection point (up to a logarithmic factor in the latter case), but also that it is able to achieve an almost-parametric rate when the true regression function is piecewise affine with not too many affine pieces. Simulations and a real data application to air pollution modelling also confirm the desirable finite-sample properties of the estimator, and our algorithm is implemented in the R package Sshaped."]]></description>
<dc:subject>to:NB regression nonparametrics optimization samworth.richard_j. carroll.raymond_j. statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:ed6d665b235e/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:optimization"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:samworth.richard_j."/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:carroll.raymond_j."/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/journals/electronic-journal-of-statistics/volume-15/issue-2/Spectrally-truncated-kernel-ridge-regression-and-its-free-lunch/10.1214/21-EJS1873.full">
    <title>Spectrally-truncated kernel ridge regression and its free lunch</title>
    <dc:date>2021-07-22T16:38:51+00:00</dc:date>
    <link>https://projecteuclid.org/journals/electronic-journal-of-statistics/volume-15/issue-2/Spectrally-truncated-kernel-ridge-regression-and-its-free-lunch/10.1214/21-EJS1873.full</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Kernel ridge regression (KRR) is a well-known and popular nonparametric regression approach with many desirable properties, including minimax rate-optimality in estimating functions that belong to common reproducing kernel Hilbert spaces (RKHS). The approach, however, is computationally intensive for large data sets, due to the need to operate on a dense n×n kernel matrix, where n is the sample size. Recently, various approximation schemes for solving KRR have been considered, and some analyzed. Some approaches such as Nyström approximation and sketching have been shown to preserve the rate optimality of KRR. In this paper, we consider the simplest approximation, namely, spectrally truncating the kernel matrix to its largest r<n eigenvalues. We derive an exact expression for the maximum risk of this truncated KRR, over the unit ball of the RKHS. This result can be used to study the exact trade-off between the level of spectral truncation and the regularization parameter. We show that, as long as the RKHS is infinite-dimensional, there is a threshold on r, above which, the spectrally-truncated KRR surprisingly outperforms the full KRR in terms of the minimax risk, where the minimum is taken over the regularization parameter. This strengthens the existing results on approximation schemes, by showing that not only one does not lose in terms of the rates, truncation can in fact improve the performance, for all finite samples (above the threshold). Moreover, we show that the implicit regularization achieved by spectral truncation is not a substitute for Hilbert norm regularization. Both are needed to achieve the best performance."

!!!]]></description>
<dc:subject>to:NB nonparametrics regression kernel_methods hilbert_space statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:79a20beb303a/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kernel_methods"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:hilbert_space"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://jmlr.org/papers/v22/19-892.html">
    <title>Adaptive estimation of nonparametric functionals</title>
    <dc:date>2021-06-07T03:48:52+00:00</dc:date>
    <link>https://jmlr.org/papers/v22/19-892.html</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We provide general adaptive upper bounds for estimating nonparametric functionals based on second-order U-statistics arising from finite-dimensional approximation of the infinite-dimensional models. We then provide examples of functionals for which the theory produces rate optimally matching adaptive upper and lower bounds. Our results are automatically adaptive in both parametric and nonparametric regimes of estimation and are automatically adaptive and semiparametric efficient in the regime of parametric convergence rate."]]></description>
<dc:subject>to:NB causal_inference nonparametrics statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:ab3ddf8ce1b4/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_inference"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://jmlr.org/papers/v22/20-1369.html">
    <title>Towards a Unified Analysis of Random Fourier Features</title>
    <dc:date>2021-06-07T02:23:38+00:00</dc:date>
    <link>https://jmlr.org/papers/v22/20-1369.html</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Random Fourier features is a widely used, simple, and effective technique for scaling up kernel methods. The existing theoretical analysis of the approach, however, remains focused on specific learning tasks and typically gives pessimistic bounds which are at odds with the empirical results. We tackle these problems and provide the first unified risk analysis of learning with random Fourier features using the squared error and Lipschitz continuous loss functions. In our bounds, the trade-off between the computational cost and the learning risk convergence rate is problem specific and expressed in terms of the regularization parameter and the number of effective degrees of freedom. We study both the standard random Fourier features method for which we improve the existing bounds on the number of features required to guarantee the corresponding minimax risk convergence rate of kernel ridge regression, as well as a data-dependent modification which samples features proportional to ridge leverage scores and further reduces the required number of features. As ridge leverage scores are expensive to compute, we devise a simple approximation scheme which provably reduces the computational cost without loss of statistical efficiency. Our empirical results illustrate the effectiveness of the proposed scheme relative to the standard random Fourier features method."

]]></description>
<dc:subject>to:NB to_read random_features nonparametrics kernel_methods statistics re:codename:catherine_wheel</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:eaf8d8877596/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:random_features"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kernel_methods"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:re:codename:catherine_wheel"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2106.01529">
    <title>[2106.01529] Minimax Optimal Regression over Sobolev Spaces via Laplacian Regularization on Neighborhood Graphs</title>
    <dc:date>2021-06-07T02:19:49+00:00</dc:date>
    <link>https://arxiv.org/abs/2106.01529</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["In this paper we study the statistical properties of Laplacian smoothing, a graph-based approach to nonparametric regression. Under standard regularity conditions, we establish upper bounds on the error of the Laplacian smoothing estimator fˆ, and a goodness-of-fit test also based on fˆ. These upper bounds match the minimax optimal estimation and testing rates of convergence over the first-order Sobolev class H1(), for ⊆ℝd and 1≤d<4; in the estimation problem, for d=4, they are optimal modulo a logn factor. Additionally, we prove that Laplacian smoothing is manifold-adaptive: if ⊆ℝd is an m-dimensional manifold with m<d, then the error rate of Laplacian smoothing (in either estimation or testing) depends only on m, in the same way it would if  were a full-dimensional set in ℝd."]]></description>
<dc:subject>to:NB smoothing nonparametrics network_data_analysis regression kith_and_kin green.alden balakrishnan.sivaraman tibshirani.ryan statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:5874c1e3bc39/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:smoothing"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:network_data_analysis"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kith_and_kin"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:green.alden"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:balakrishnan.sivaraman"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:tibshirani.ryan"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2105.15197">
    <title>[2105.15197] A Simple and General Debiased Machine Learning Theorem with Finite Sample Guarantees</title>
    <dc:date>2021-06-01T13:38:00+00:00</dc:date>
    <link>https://arxiv.org/abs/2105.15197</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Debiased machine learning is a meta algorithm based on bias correction and sample splitting to calculate confidence intervals for functionals (i.e. scalar summaries) of machine learning algorithms. For example, an analyst may desire the confidence interval for a treatment effect estimated with a neural network. We provide a nonasymptotic debiased machine learning theorem that encompasses any global or local functional of any machine learning algorithm that satisfies a few simple, interpretable conditions. Formally, we prove consistency, Gaussian approximation, and semiparametric efficiency by finite sample arguments. The rate of convergence is root-n for global functionals, and it degrades gracefully for local functionals. Our results culminate in a simple set of conditions that an analyst can use to translate modern learning theory rates into traditional statistical inference. The conditions reveal a new double robustness property for ill posed inverse problems."]]></description>
<dc:subject>to:NB confidence_sets nonparametrics to_read statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:a2a745b5e206/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:confidence_sets"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2105.14075">
    <title>[2105.14075] Distribution-free inference for regression: discrete, continuous, and in between</title>
    <dc:date>2021-06-01T13:34:48+00:00</dc:date>
    <link>https://arxiv.org/abs/2105.14075</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["In data analysis problems where we are not able to rely on distributional assumptions, what types of inference guarantees can still be obtained? Many popular methods, such as holdout methods, cross-validation methods, and conformal prediction, are able to provide distribution-free guarantees for predictive inference, but the problem of providing inference for the underlying regression function (for example, inference on the conditional mean 𝔼[Y|X]) is more challenging. In the setting where the features X are continuously distributed, recent work has established that any confidence interval for 𝔼[Y|X] must have non-vanishing width, even as sample size tends to infinity. At the other extreme, if X takes only a small number of possible values, then inference on 𝔼[Y|X] is trivial to achieve. In this work, we study the problem in settings in between these two extremes. We find that there are several distinct regimes in between the finite setting and the continuous setting, where vanishing-width confidence intervals are achievable if and only if the effective support size of the distribution of X is smaller than the square of the sample size."]]></description>
<dc:subject>to:NB regression confidence_sets nonparametrics barber.rina_foygel statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:31da98979df1/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:confidence_sets"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:barber.rina_foygel"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1904.00989">
    <title>[1904.00989] Counterfactual Sensitivity and Robustness</title>
    <dc:date>2021-05-18T18:00:43+00:00</dc:date>
    <link>https://arxiv.org/abs/1904.00989</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We propose a framework for characterizing the sensitivity of counterfactuals with respect to parametric assumptions about the distribution of latent variables in a class of structural models. In particular, we show how to characterize the smallest and largest values of the counterfactual as the distribution of latent variables spans nonparametric neighborhoods of a researcher's parametric specification while other "structural" features of the model are maintained. Our procedure replaces the infinite-dimensional optimization with respect to the distribution by a finite-dimensional convex program and is therefore computationally simple to implement. We develop a novel MPEC implementation of our procedure to further simplify computation in models featuring endogenous parameters defined by equilibrium constraints. Our procedure recovers sharp bounds on the nonparametrically identified set of counterfactuals over large neighborhoods and has connections with local approaches to sensitivity analysis over small neighborhoods. We propose plug-in estimators of the smallest and largest counterfactuals and two procedures for inference. We illustrate the broad applicability of our procedure with empirical applications to matching models and dynamic discrete choice."]]></description>
<dc:subject>to:NB causal_inference nonparametrics sensitivity_analysis model_checking statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:a58a88c92b64/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_inference"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:sensitivity_analysis"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:model_checking"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2105.04754">
    <title>[2105.04754] Non-Parametric Estimation of Manifolds from Noisy Data</title>
    <dc:date>2021-05-12T18:21:01+00:00</dc:date>
    <link>https://arxiv.org/abs/2105.04754</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["A common observation in data-driven applications is that high dimensional data has a low intrinsic dimension, at least locally. In this work, we consider the problem of estimating a d dimensional sub-manifold of ℝD from a finite set of noisy samples. Assuming that the data was sampled uniformly from a tubular neighborhood of ∈k, a compact manifold without boundary, we present an algorithm that takes a point r from the tubular neighborhood and outputs p̂ n∈ℝD, and Tp̂ nˆ an element in the Grassmanian Gr(d,D). We prove that as the number of samples n→∞ the point p̂ n converges to p∈ and Tp̂ nˆ converges to Tp (the tangent space at that point) with high probability. Furthermore, we show that the estimation yields asymptotic rates of convergence of n−k2k+d for the point estimation and n−k−12k+d for the estimation of the tangent space. These rates are known to be optimal for the case of function estimation."]]></description>
<dc:subject>to:NB manifold_learning nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:d2b1a4fadc3e/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:manifold_learning"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2012.13106">
    <title>[2012.13106] Dependence of variance on covariate design in nonparametric link regression</title>
    <dc:date>2021-05-12T18:12:37+00:00</dc:date>
    <link>https://arxiv.org/abs/2012.13106</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["This paper discusses a nonparametric approach to link regression aiming at predicting a mean outcome at a link, i.e., a pair of nodes, based on currently observed data comprising covariates at nodes and outcomes at links. The variance decay rates of nonparametric link regression estimates are demonstrated to depend on covariate designs; namely, whether the covariate design is random or fixed. This covariate design-dependent nature of variance is observed in nonparametric link regression but not in conventional nonparametric regression."]]></description>
<dc:subject>to:NB network_data_analysis regression nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:32e8bc8f0998/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:network_data_analysis"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2104.10601">
    <title>[2104.10601] Statistical inference for generative adversarial networks</title>
    <dc:date>2021-04-22T15:20:39+00:00</dc:date>
    <link>https://arxiv.org/abs/2104.10601</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["This paper studies generative adversarial networks (GANs) from a statistical perspective. A GAN is a popular machine learning method in which the parameters of two neural networks, a generator and a discriminator, are estimated to solve a particular minimax problem. This minimax problem typically has a multitude of solutions and the focus of this paper are the statistical properties of these solutions. We address two key issues for the generator and discriminator network parameters, consistent estimation and confidence sets. We first show that the set of solutions to the sample GAN problem is a (Hausdorff) consistent estimator of the set of solutions to the corresponding population GAN problem. We then devise a computationally intensive procedure to form confidence sets and show that these sets contain the population GAN solutions with the desired coverage probability. The assumptions employed in our results are weak and hold in many practical GAN applications. To the best of our knowledge, this paper provides the first results on statistical inference for GANs in the empirically relevant case of multiple solutions."]]></description>
<dc:subject>neural_networks confidence_sets nonparametrics your_favorite_deep_neural_network_sucks in_NB</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:6f7375fd36ee/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:neural_networks"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:confidence_sets"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:your_favorite_deep_neural_network_sucks"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2010.04855">
    <title>[2010.04855] Reproducing Kernel Methods for Nonparametric and Semiparametric Treatment Effects</title>
    <dc:date>2021-04-22T15:18:05+00:00</dc:date>
    <link>https://arxiv.org/abs/2010.04855</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We propose a family of reproducing kernel ridge estimators for nonparametric and semiparametric policy evaluation. The framework includes (i) treatment effects of the population, of subpopulations, and of alternative populations; (ii) the decomposition of a total effect into a direct effect and an indirect effect (mediated by a particular mechanism); and (iii) effects of sequences of treatments. Treatment and covariates may be discrete or continuous, and low, high, or infinite dimensional. We consider estimation of means, increments, and distributions of counterfactual outcomes. Each estimator is an inner product in a reproducing kernel Hilbert space (RKHS), with a one line, closed form solution. For the nonparametric case, we prove uniform consistency and provide finite sample rates of convergence. For the semiparametric case, we prove root n consistency, Gaussian approximation, and semiparametric efficiency by finite sample arguments. We evaluate our estimators in simulations then estimate continuous, heterogeneous, incremental, and mediated treatment effects of the US Jobs Corps training program for disadvantaged youth."]]></description>
<dc:subject>to:NB causal_inference nonparametrics kernel_methods</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:44a4baf7ac02/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_inference"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kernel_methods"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2104.09935">
    <title>[2104.09935] CATE meets ML - The Conditional Average Treatment Effect and Machine Learning</title>
    <dc:date>2021-04-21T19:55:02+00:00</dc:date>
    <link>https://arxiv.org/abs/2104.09935</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["For treatment effects - one of the core issues in modern econometric analysis - prediction and estimation are two sides of the same coin. As it turns out, machine learning methods are the tool for generalized prediction models. Combined with econometric theory, they allow us to estimate not only the average but a personalized treatment effect - the conditional average treatment effect (CATE). In this tutorial, we give an overview of novel methods, explain them in detail, and apply them via Quantlets in real data applications. We study the effect that microcredit availability has on the amount of money borrowed and if 401(k) pension plan eligibility has an impact on net financial assets, as two empirical examples. The presented toolbox of methods contains meta-learners, like the Doubly-Robust, R-, T- and X-learner, and methods that are specially designed to estimate the CATE like the causal BART and the generalized random forest. In both, the microcredit and 401(k) example, we find a positive treatment effect for all observations but conflicting evidence of treatment effect heterogeneity. An additional simulation study, where the true treatment effect is known, allows us to compare the different methods and to observe patterns and similarities."]]></description>
<dc:subject>to:NB causal_inference nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:c2795042cb03/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_inference"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1406.5362">
    <title>[1406.5362] Predicting the Future Behavior of a Time-Varying Probability Distribution</title>
    <dc:date>2021-04-21T19:54:13+00:00</dc:date>
    <link>https://arxiv.org/abs/1406.5362</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We study the problem of predicting the future, though only in the probabilistic sense of estimating a future state of a time-varying probability distribution. This is not only an interesting academic problem, but solving this extrapolation problem also has many practical application, e.g. for training classifiers that have to operate under time-varying conditions. Our main contribution is a method for predicting the next step of the time-varying distribution from a given sequence of sample sets from earlier time steps. For this we rely on two recent machine learning techniques: embedding probability distributions into a reproducing kernel Hilbert space, and learning operators by vector-valued regression. We illustrate the working principles and the practical usefulness of our method by experiments on synthetic and real data. We also highlight an exemplary application: training a classifier in a domain adaptation setting without having access to examples from the test time distribution at training time."]]></description>
<dc:subject>to:NB prediction probability hilbert_space nonparametrics re:AoS_project</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:aa1867d0c690/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:prediction"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:probability"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:hilbert_space"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:re:AoS_project"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2004.14497">
    <title>[2004.14497] Optimal doubly robust estimation of heterogeneous causal effects</title>
    <dc:date>2021-04-21T19:44:40+00:00</dc:date>
    <link>https://arxiv.org/abs/2004.14497</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Heterogeneous effect estimation plays a crucial role in causal inference, with applications across medicine and social science. Many methods for estimating conditional average treatment effects (CATEs) have been proposed in recent years, but there are important theoretical gaps in understanding if and when such methods are optimal. This is especially true when the CATE has nontrivial structure (e.g., smoothness or sparsity). Our work contributes in several main ways. First, we study a two-stage doubly robust CATE estimator and give a generic model-free error bound, which, despite its generality, yields sharper results than those in the current literature. We apply the bound to derive error rates in nonparametric models with smoothness or sparsity, and give sufficient conditions for oracle efficiency. Underlying our error bound is a general oracle inequality for regression with estimated or imputed outcomes, which is of independent interest; this is the second main contribution. The third contribution is aimed at understanding the fundamental statistical limits of CATE estimation. To that end, we propose and study a local polynomial adaptation of double-residual regression. We show that this estimator can be oracle efficient under even weaker conditions, if used with a specialized form of sample splitting and careful choices of tuning parameters. These are the weakest conditions currently found in the literature, and we conjecture that they are minimal in a minimax sense. We go on to give error bounds in the non-trivial regime where oracle rates cannot be achieved. Some finite-sample properties are explored with simulations."]]></description>
<dc:subject>causal_inference nonparametrics heard_the_talk have_read in_NB kennedy.edward_h.</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:e1f9f232f51d/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_inference"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:heard_the_talk"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:have_read"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kennedy.edward_h."/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://www.tandfonline.com/doi/full/10.1080/01621459.2021.1895175">
    <title>Consistent Sparse Deep Learning: Theory and Computation: Journal of the American Statistical Association: Vol 0, No 0</title>
    <dc:date>2021-04-21T16:09:37+00:00</dc:date>
    <link>https://www.tandfonline.com/doi/full/10.1080/01621459.2021.1895175</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Deep learning has been the engine powering many successes of data science. However, the deep neural network (DNN), as the basic model of deep learning, is often excessively over-parameterized, causing many difficulties in training, prediction and interpretation. We propose a frequentist-like method for learning sparse DNNs and justify its consistency under the Bayesian framework: the proposed method could learn a sparse DNN with at most O(n/log(n))O(n/ log (n)) connections and nice theoretical guarantees such as posterior consistency, variable selection consistency and asymptotically optimal generalization bounds. In particular, we establish posterior consistency for the sparse DNN with a mixture Gaussian prior, show that the structure of the sparse DNN can be consistently determined using a Laplace approximation-based marginal posterior inclusion probability approach, and use Bayesian evidence to elicit sparse DNNs learned by an optimization method such as stochastic gradient descent in multiple runs with different initializations. The proposed method is computationally more efficient than standard Bayesian methods for large-scale sparse DNNs. The numerical results indicate that the proposed method can perform very well for large-scale network compression and high-dimensional nonlinear variable selection, both advancing interpretable machine learning."]]></description>
<dc:subject>neural_networks nonparametrics sparsity bayesian_consistency in_NB</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:eaf86c3cae34/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:neural_networks"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:sparsity"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:bayesian_consistency"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1802.08667">
    <title>[1802.08667] De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers</title>
    <dc:date>2021-04-14T14:48:58+00:00</dc:date>
    <link>https://arxiv.org/abs/1802.08667</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We provide adaptive inference methods, based on ℓ1 regularization, for regular (semiparametric) and non-regular (nonparametric) linear functionals of the conditional expectation function. Examples of regular functionals include average treatment effects, policy effects, and derivatives. Examples of non-regular functionals include average treatment effects, policy effects, and derivatives conditional on a covariate subvector fixed at a point. We construct a Neyman orthogonal equation for the target parameter that is approximately invariant to small perturbations of the nuisance parameters. To achieve this property, we include the Riesz representer for the functional as an additional nuisance parameter. Our analysis yields weak "double sparsity robustness": either the approximation to the regression or the approximation to the representer can be "completely dense" as long as the other is sufficiently "sparse". Our main results are non-asymptotic and imply asymptotic uniform validity over large classes of models, translating into honest confidence bands for both global and local parameters."]]></description>
<dc:subject>to:NB causal_inference sparsity nonparametrics to_read</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:56b3f2bb1d9f/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_inference"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:sparsity"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1809.05224">
    <title>[1809.05224] Automatic Debiased Machine Learning of Causal and Structural Effects</title>
    <dc:date>2021-04-14T14:48:17+00:00</dc:date>
    <link>https://arxiv.org/abs/1809.05224</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Many causal and structural effects depend on regressions. Examples include policy effects, average derivatives, regression decompositions, average treatment effects, causal mediation, and parameters of economic structural models. The regressions may be high dimensional, making machine learning useful. Plugging machine learners into identifying equations can lead to poor inference due to bias from regularization and/or model selection. This paper gives automatic debiasing for linear and nonlinear functions of regressions. The debiasing is automatic in using Lasso and the function of interest without the full form of the bias correction. The debiasing can be applied to any regression learner, including neural nets, random forests, Lasso, boosting, and other high dimensional methods. In addition to providing the bias correction we give standard errors that are robust to misspecification, convergence rates for the bias correction, and primitive conditions for asymptotic inference for estimators of a variety of estimators of structural and causal effects. The automatic debiased machine learning is used to estimate the average treatment effect on the treated for the NSW job training data and to estimate demand elasticities from Nielsen scanner data while allowing preferences to be correlated with prices and income."]]></description>
<dc:subject>to:NB causal_inference nonparametrics to_read</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:00926a601fb1/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_inference"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://onlinelibrary.wiley.com/doi/10.1111/sjos.12529">
    <title>Factorized Estimation of High‐Dimensional Nonparametric Covariance Models - Zhang - - Scandinavian Journal of Statistics - Wiley Online Library</title>
    <dc:date>2021-04-12T03:44:07+00:00</dc:date>
    <link>https://onlinelibrary.wiley.com/doi/10.1111/sjos.12529</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Estimation of covariate‐dependent conditional covariance matrix in a high‐dimensional space poses a challenge to contemporary statistical research. The existing kernel estimators may not be locally adaptive due to using a single bandwidth to explore the smoothness of all entries of the target matrix function. In this paper, we propose a novel framework to address this issue, where we factorize the target matrix into factors and estimate these factors in turn by the kernel approach. The resulting estimator is further regularized by thresholding and optimal shrinkage. Under certain mixing and sparsity conditions, we show that the proposed estimator is well‐conditioned and uniformly consistent with the underlying matrix even when the sample is dependent. Simulation studies suggest that the proposed estimator significantly outperforms its competitors in terms of integrated root‐squared estimation error. We present an application to financial return data."]]></description>
<dc:subject>to:NB variance_estimation nonparametrics statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:d4c95ff04060/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:variance_estimation"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2101.10255">
    <title>[2101.10255] Consistent specification testing under spatial dependence</title>
    <dc:date>2021-01-26T05:47:28+00:00</dc:date>
    <link>https://arxiv.org/abs/2101.10255</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We propose a series-based nonparametric specification test for a regression function when data are spatially dependent, the `space' being of a general economic or social nature. Dependence can be parametric, parametric with increasing dimension, semiparametric or any combination thereof, thus covering a vast variety of settings. These include spatial error models of varying types and levels of complexity. Under a new smooth spatial dependence condition, our test statistic is asymptotically standard normal. To prove the latter property, we establish a central limit theorem for quadratic forms in linear processes in an increasing dimension setting. Finite sample performance is investigated in a simulation study and empirical examples illustrate the test with real-world data."]]></description>
<dc:subject>to:NB spatial_statistics misspecification statistics regression nonparametrics to_read to_teach:data_over_space_and_time</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:69cd6ce3b5cf/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:spatial_statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:misspecification"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_teach:data_over_space_and_time"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1806.05161">
    <title>[1806.05161] Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate</title>
    <dc:date>2021-01-22T20:14:51+00:00</dc:date>
    <link>https://arxiv.org/abs/1806.05161</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Many modern machine learning models are trained to achieve zero or near-zero training error in order to obtain near-optimal (but non-zero) test error. This phenomenon of strong generalization performance for "overfitted" / interpolated classifiers appears to be ubiquitous in high-dimensional data, having been observed in deep networks, kernel machines, boosting and random forests. Their performance is consistently robust even when the data contain large amounts of label noise.
"Very little theory is available to explain these observations. The vast majority of theoretical analyses of generalization allows for interpolation only when there is little or no label noise. This paper takes a step toward a theoretical foundation for interpolated classifiers by analyzing local interpolating schemes, including geometric simplicial interpolation algorithm and singularly weighted k-nearest neighbor schemes. Consistency or near-consistency is proved for these schemes in classification and regression problems. Moreover, the nearest neighbor schemes exhibit optimal rates under some standard statistical assumptions.
"Finally, this paper suggests a way to explain the phenomenon of adversarial examples, which are seemingly ubiquitous in modern machine learning, and also discusses some connections to kernel machines and random forests in the interpolated regime."
]]></description>
<dc:subject>learning_theory nonparametrics minimax belkin.mikhail hsu.daniel to_read to_teach:childs_garden_of_statistical_learning_theory in_NB interpolation_aka_memorizing_the_training_data</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:14df95f530fb/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:minimax"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:belkin.mikhail"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:hsu.daniel"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_teach:childs_garden_of_statistical_learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:interpolation_aka_memorizing_the_training_data"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1806.09471">
    <title>[1806.09471] Does data interpolation contradict statistical optimality?</title>
    <dc:date>2021-01-22T20:12:52+00:00</dc:date>
    <link>https://arxiv.org/abs/1806.09471</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We show that learning methods interpolating the training data can achieve optimal rates for the problems of nonparametric regression and prediction with square loss."]]></description>
<dc:subject>learning_theory nonparametrics minimax belkin.mikhail rakhlin.alexander to_teach:childs_garden_of_statistical_learning_theory tsybakov.alexandre_b. have_read in_NB interpolation_aka_memorizing_the_training_data</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:299dc566e146/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:minimax"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:belkin.mikhail"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:rakhlin.alexander"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_teach:childs_garden_of_statistical_learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:tsybakov.alexandre_b."/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:have_read"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:interpolation_aka_memorizing_the_training_data"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2101.04783">
    <title>[2101.04783] Variable bandwidth kernel regression estimation</title>
    <dc:date>2021-01-14T15:59:57+00:00</dc:date>
    <link>https://arxiv.org/abs/2101.04783</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["In this paper we propose a variable bandwidth kernel regression estimator for i.i.d. observations in ℝ2 to improve the classical Nadaraya-Watson estimator. The bias is improved to the order of O(h4n) under the condition that the fifth order derivative of the density function and the sixth order derivative of the regression function are bounded and continuous. We also establish the central limit theorems for the proposed ideal and true variable kernel regression estimators. The simulation study confirms our results and demonstrates the advantage of the variable bandwidth kernel method over the classical kernel method."

--- Didn't Silverman show back in the '80s that if you carry this line of thought through to the logical conclusion, you get a smoothing spline?]]></description>
<dc:subject>to:NB smoothing regression nonparametrics kernel_smoothing</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:bbc254231a9c/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:smoothing"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kernel_smoothing"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://www.pnas.org/content/113/38/10530">
    <title>Extracting multistage screening rules from online dating activity data | PNAS</title>
    <dc:date>2021-01-07T21:14:16+00:00</dc:date>
    <link>https://www.pnas.org/content/113/38/10530</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["This paper presents a statistical framework for harnessing online activity data to better understand how people make decisions. Building on insights from cognitive science and decision theory, we develop a discrete choice model that allows for exploratory behavior and multiple stages of decision making, with different rules enacted at each stage. Critically, the approach can identify if and when people invoke noncompensatory screeners that eliminate large swaths of alternatives from detailed consideration. The model is estimated using deidentified activity data on 1.1 million browsing and writing decisions observed on an online dating site. We find that mate seekers enact screeners (“deal breakers”) that encode acceptability cutoffs. A nonparametric account of heterogeneity reveals that, even after controlling for a host of observable attributes, mate evaluation differs across decision stages as well as across identified groupings of men and women. Our statistical framework can be widely applied in analyzing large-scale data on multistage choices, which typify searches for “big ticket” items."]]></description>
<dc:subject>to:NB decision-making statistics nonparametrics mixture_models practices_relating_to_the_transmission_of_genetic_information to_teach:undergrad-ADA via:gabriel_rossman feinberg.fred</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:a8ecf81231c6/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:decision-making"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:mixture_models"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:practices_relating_to_the_transmission_of_genetic_information"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_teach:undergrad-ADA"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:via:gabriel_rossman"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:feinberg.fred"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2101.01535">
    <title>[2101.01535] An RKHS-Based Semiparametric Approach to Nonlinear Sufficient Dimension Reduction</title>
    <dc:date>2021-01-06T17:10:25+00:00</dc:date>
    <link>https://arxiv.org/abs/2101.01535</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Based on the theory of reproducing kernel Hilbert space (RKHS) and semiparametric method, we propose a new approach to nonlinear dimension reduction. The method extends the semiparametric method into a more generalized domain where both the interested parameters and nuisance parameters to be infinite dimensional. By casting the nonlinear dimensional reduction problem in a generalized semiparametric framework, we calculate the orthogonal complement space of generalized nuisance tangent space to derive the estimating equation. Solving the estimating equation by the theory of RKHS and regularization, we obtain the estimation of dimension reduction directions of the sufficient dimension reduction (SDR) subspace and also show the asymptotic property of estimator. Furthermore, the proposed method does not rely on the linearity condition and constant variance condition. Simulation and real data studies are conducted to demonstrate the finite sample performance of our method in comparison with several existing methods."]]></description>
<dc:subject>to:NB nonparametrics hilbert_space dimension_reduction statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:4499fc48f267/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:hilbert_space"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:dimension_reduction"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2012.14482">
    <title>[2012.14482] Multivariate Smoothing via the Fourier Integral Theorem and Fourier Kernel</title>
    <dc:date>2021-01-03T20:13:12+00:00</dc:date>
    <link>https://arxiv.org/abs/2012.14482</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Starting with the Fourier integral theorem, we present natural Monte Carlo estimators of multivariate functions including densities, mixing densities, transition densities, regression functions, and the search for modes of multivariate density functions (modal regression). Rates of convergence are established and, in many cases, provide superior rates to current standard estimators such as those based on kernels, including kernel density estimators and kernel regression functions. Numerical illustrations are presented."]]></description>
<dc:subject>to:NB fourier_analysis smoothing computational_statistics density_estimation nonparametrics regression statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:7b370340cb7a/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:fourier_analysis"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:smoothing"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:computational_statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:density_estimation"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2012.14563">
    <title>[2012.14563] Random Planted Forest: a directly interpretable tree ensemble</title>
    <dc:date>2021-01-03T20:06:28+00:00</dc:date>
    <link>https://arxiv.org/abs/2012.14563</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We introduce a novel interpretable and tree-based algorithm for prediction in a regression setting in which each tree in a classical random forest is replaced by a family of planted trees that grow simultaneously. The motivation for our algorithm is to estimate the unknown regression function from a functional ANOVA decomposition perspective, where each tree corresponds to a function within that decomposition. Therefore, planted trees are limited in the number of interaction terms. The maximal order of approximation in the ANOVA decomposition can be specified or left unlimited. If a first order approximation is chosen, the result is an additive model. In the other extreme case, if the order of approximation is not limited, the resulting model puts no restrictions on the form of the regression function. In a simulation study we find encouraging prediction and visualisation properties of our random planted forest method. We also develop theory for an idealised version of random planted forests in the case of an underlying additive model. We show that in the additive case, the idealised version achieves up to a logarithmic factor asymptotically optimal one-dimensional convergence rates of order n−2/5."]]></description>
<dc:subject>to:NB regression nonparametrics ensemble_methods decision_trees to_teach:data-mining</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:f6e4ba73e30f/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:ensemble_methods"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:decision_trees"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_teach:data-mining"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2006.11937">
    <title>[2006.11937] Learning of Discrete Graphical Models with Neural Networks</title>
    <dc:date>2020-12-26T17:46:11+00:00</dc:date>
    <link>https://arxiv.org/abs/2006.11937</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Graphical models are widely used in science to represent joint probability distributions with an underlying conditional dependence structure. The inverse problem of learning a discrete graphical model given i.i.d samples from its joint distribution can be solved with near-optimal sample complexity using a convex optimization method known as Generalized Regularized Interaction Screening Estimator (GRISE). But the computational cost of GRISE becomes prohibitive when the energy function of the true graphical model has higher-order terms. We introduce NeurISE, a neural net based algorithm for graphical model learning, to tackle this limitation of GRISE. We use neural nets as function approximators in an Interaction Screening objective function. The optimization of this objective then produces a neural-net representation for the conditionals of the graphical model. NeurISE algorithm is seen to be a better alternative to GRISE when the energy function of the true model has a high order with a high degree of symmetry. In these cases NeurISE is able to find the correct parsimonious representation for the conditionals without being fed any prior information about the true model. NeurISE can also be used to learn the underlying structure of the true model with some simple modifications to its training procedure. In addition, we also show a variant of NeurISE that can be used to learn a neural net representation for the full energy function of the true model."

--- Comment before reading: I'm willing to bet it's the "high degree of symmetry" that's doing the work, and not any magic of neural networks.  But I could be wrong!]]></description>
<dc:subject>to:NB graphical_models causal_discovery nonparametrics statistics neural_networks</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:db060342bc93/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:graphical_models"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_discovery"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:neural_networks"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/euclid.ss/1608541216">
    <title>Gao , Ma : Minimax Rates in Network Analysis: Graphon Estimation, Community Detection and Hypothesis Testing</title>
    <dc:date>2020-12-21T14:11:01+00:00</dc:date>
    <link>https://projecteuclid.org/euclid.ss/1608541216</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["This paper surveys some recent developments in fundamental limits and optimal algorithms for network analysis. We focus on minimax optimal rates in three fundamental problems of network analysis: graphon estimation, community detection and hypothesis testing. For each problem, we review state-of-the-art results in the literature followed by general principles behind the optimal procedures that lead to minimax estimation and testing. This allows us to connect problems in network analysis to other statistical inference problems from a general perspective."]]></description>
<dc:subject>to:NB network_data_analysis graph_limits hypothesis_testing minimax nonparametrics re:smoothing_adjacency_matrices community_discovery to_read</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:4146f1114577/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:network_data_analysis"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:graph_limits"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:hypothesis_testing"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:minimax"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:re:smoothing_adjacency_matrices"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:community_discovery"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2007.04286">
    <title>[2007.04286] Kernel-based Prediction of Non-Markovian Time Series</title>
    <dc:date>2020-12-17T20:50:05+00:00</dc:date>
    <link>https://arxiv.org/abs/2007.04286</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["A nonparametric method to predict non-Markovian time series of partially observed dynamics is developed. The prediction problem we consider is a supervised learning task of finding a regression function that takes a delay embedded observable to the observable at a future time. When delay embedding theory is applicable, the proposed regression function is a consistent estimator of the flow map induced by the delay embedding. Furthermore, the corresponding Mori-Zwanzig equation governing the evolution of the observable simplifies to only a Markovian term, represented by the regression function. We realize this supervised learning task with a class of kernel-based linear estimators, the kernel analog forecast (KAF), which are consistent in the limit of large data. In a scenario with a high-dimensional covariate space, we employ a Markovian kernel smoothing method which is computationally cheaper than the Nyström projection method for realizing KAF. In addition to the guaranteed theoretical convergence, we numerically demonstrate the effectiveness of this approach on higher-dimensional problems where the relevant kernel features are difficult to capture with the Nyström method. Given noisy training data, we propose a nonparametric smoother as a de-noising method. Numerically, we show that the proposed smoother is more accurate than EnKF and 4Dvar in de-noising signals corrupted by independent (but not necessarily identically distributed) noise, even if the smoother is constructed using a data set corrupted by white noise. We show skillful prediction using the KAF constructed from the denoised data."]]></description>
<dc:subject>to:NB time_series regression prediction statistics nonparametrics kernel_smoothing</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:8dd8fb122f34/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:time_series"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:prediction"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kernel_smoothing"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://academic.oup.com/biomet/article-abstract/107/4/983/5857283?redirectedFrom=fulltext">
    <title>unified approach to the calculation of information operators in semiparametric models | Biometrika | Oxford Academic</title>
    <dc:date>2020-12-17T01:41:19+00:00</dc:date>
    <link>https://academic.oup.com/biomet/article-abstract/107/4/983/5857283?redirectedFrom=fulltext</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["The infinite-dimensional information operator for the nuisance parameter plays a key role in semiparametric inference, as it is closely related to the regular estimability of the target parameter. Calculation of information operators has traditionally proceeded in a case-by-case manner and has often entailed lengthy derivations with complicated arguments. We develop a unified framework for this task by exploiting commonality in the form of semiparametric likelihoods. The general formula developed allows one to derive information operators with simple calculus and, if necessary at all, a minimal amount of probabilistic evaluation. This streamlined approach shows its simplicity and versatility in application to a number of existing models as well as a new model of practical interest."]]></description>
<dc:subject>to:NB statistics nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:fce82a81bbf6/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2005.06371">
    <title>[2005.06371] Nonparametric regression for locally stationary random fields under stochastic sampling design</title>
    <dc:date>2020-12-16T17:47:23+00:00</dc:date>
    <link>https://arxiv.org/abs/2005.06371</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["In this study, we develop an asymptotic theory of nonparametric regression for locally stationary random fields (LSRFs) {Xs,An:s∈Rn} in ℝp observed at irregularly spaced locations in Rn=[0,An]d⊂ℝd. We first derive the uniform convergence rate of general kernel estimators, followed by the asymptotic normality of an estimator for the mean function of the model. Moreover, we consider additive models to avoid the curse of dimensionality arising from the dependence of the convergence rate of estimators on the number of covariates. Subsequently, we derive the uniform convergence rate and joint asymptotic normality of the estimators for additive functions. We also introduce approximately mn-dependent RFs to provide examples of LSRFs. We find that these RFs include a wide class of Lévy-driven moving average RFs."]]></description>
<dc:subject>to:NB random_fields spatial_statistics spatio-temporal_statistics smoothing nonparametrics regression additive_models statistics to_teach:data_over_space_and_time kernel_smoothing</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:9759167da0d4/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:random_fields"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:spatial_statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:spatio-temporal_statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:smoothing"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:additive_models"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_teach:data_over_space_and_time"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kernel_smoothing"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2012.08444">
    <title>[2012.08444] Minimax Risk and Uniform Convergence Rates for Nonparametric Dyadic Regression</title>
    <dc:date>2020-12-16T15:11:23+00:00</dc:date>
    <link>https://arxiv.org/abs/2012.08444</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Let i=1,…,N index a simple random sample of units drawn from some large population. For each unit we observe the vector of regressors Xi and, for each of the N(N−1) ordered pairs of units, an outcome Yij. The outcomes Yij and Ykl are independent if their indices are disjoint, but dependent otherwise (i.e., "dyadically dependent"). Let Wij=(X′i,X′j)′; using the sampled data we seek to construct a nonparametric estimate of the mean regression function g(Wij)≡def𝔼[Yij∣∣Xi,Xj].
"We present two sets of results. First, we calculate lower bounds on the minimax risk for estimating the regression function at (i) a point and (ii) under the infinity norm. Second, we calculate (i) pointwise and (ii) uniform convergence rates for the dyadic analog of the familiar Nadaraya-Watson (NW) kernel regression estimator. We show that the NW kernel regression estimator achieves the optimal rates suggested by our risk bounds when an appropriate bandwidth sequence is chosen. This optimal rate differs from the one available under iid data: the effective sample size is smaller and dW=dim(Wij) influences the rate differently."]]></description>
<dc:subject>to:NB network_data_analysis regression nonparametrics re:smoothing_adjacency_matrices to_read</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:de76b10c1394/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:network_data_analysis"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:re:smoothing_adjacency_matrices"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2012.03376">
    <title>[2012.03376] A Lecture About the Use of Orlicz Spaces in Information Geometry</title>
    <dc:date>2020-12-09T18:48:27+00:00</dc:date>
    <link>https://arxiv.org/abs/2012.03376</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["A tutorial about Non-parametric Information Geometry, Statistical bundles, Orlicz spaces, and Gaussian Orlicz-Sobolev spaces."]]></description>
<dc:subject>to:NB statistics information_geometry nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:d2b76a0b7bf0/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:information_geometry"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2011.12215">
    <title>[2011.12215] Searching for Interactions: Why the Laplace Kernel is your Friend</title>
    <dc:date>2020-11-25T14:25:54+00:00</dc:date>
    <link>https://arxiv.org/abs/2011.12215</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We tackle the problem of nonparametric variable selection with a focus on discovering interactions between variables. With p variables there are O(ps) possible order-s interactions making exhaustive search infeasible. It is nonetheless possible to identify the variables involved in interactions with only linear computation cost, O(p). The trick is to maximize a class of parametrized nonparametric dependence measures which we call \emph{metric learning objectives}; the landscape of these nonconvex objective functions is sensitive to interactions but the objectives themselves do not explicitly model interactions. Three properties make metric learning objectives highly attractive:
"(a) The stationary points of the objective are automatically sparse (i.e. performs selection)---no explicit ℓ1 penalization is needed.
"(b) All stationary points of the objective exclude noise variables with high probability.
"(c) Guaranteed recovery of all signal variables without needing to reach the objective's global maxima or special stationary points.
"The second and third properties mean that all our theoretical results apply in the practical case where one uses gradient ascent to maximize the metric learning objective. While not all metric learning objectives enjoy good statistical power, we design an objective based on ℓ1 kernels that does exhibit favorable power: it recovers (i) main effects with n∼logp samples, (ii) hierarchical interactions with n∼logp samples and (iii) order-s pure interactions with n∼p2(s−1)logp samples."]]></description>
<dc:subject>to:NB sparsity nonparametrics statistics kernel_methods</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:2c6e8e00596e/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:sparsity"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kernel_methods"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1712.00038">
    <title>[1712.00038] Augmented Minimax Linear Estimation</title>
    <dc:date>2020-11-23T17:43:08+00:00</dc:date>
    <link>https://arxiv.org/abs/1712.00038</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Many statistical estimands can expressed as continuous linear functionals of a conditional expectation function. This includes the average treatment effect under unconfoundedness and generalizations for continuous-valued and personalized treatments. In this paper, we discuss a general approach to estimating such quantities: we begin with a simple plug-in estimator based on an estimate of the conditional expectation function, and then correct the plug-in estimator by subtracting a minimax linear estimate of its error. We show that our method is semiparametrically efficient under weak conditions and observe promising performance on both real and simulated data."]]></description>
<dc:subject>to:NB causal_inference nonparametrics statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:26f6c8f0c2be/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:causal_inference"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://www.annualreviews.org/doi/abs/10.1146/annurev-control-053018-023744">
    <title>System Identification: A Machine Learning Perspective | Annual Review of Control, Robotics, and Autonomous Systems</title>
    <dc:date>2020-11-19T05:21:49+00:00</dc:date>
    <link>https://www.annualreviews.org/doi/abs/10.1146/annurev-control-053018-023744</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Estimation of functions from sparse and noisy data is a central theme in machine learning. In the last few years, many algorithms have been developed that exploit Tikhonov regularization theory and reproducing kernel Hilbert spaces. These are the so-called kernel-based methods, which include powerful approaches like regularization networks, support vector machines, and Gaussian regression. Recently, these techniques have also gained popularity in the system identification community. In both linear and nonlinear settings, kernels that incorporate information on dynamic systems, such as the smoothness and stability of the input–output map, can challenge consolidated approaches based on parametric model structures. In the classical parametric setting, the complexity of the model (the model order) needs to be chosen, typically from a finite family of alternatives, by trading bias and variance. This (discrete) model order selection step may be critical, especially when the true model does not belong to the model class. In regularization-based approaches, model complexity is controlled by tuning (continuous) regularization parameters, making the model selection step more robust. In this article, we review these new kernel-based system identification approaches and discuss extensions based on nuclear and  norms."]]></description>
<dc:subject>to:NB nonparametrics statistics learning_theory dynamical_systems to_teach:childs_garden_of_statistical_learning_theory learning_under_dependence</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:72d9379f7c5e/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:dynamical_systems"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_teach:childs_garden_of_statistical_learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:learning_under_dependence"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/euclid.aos/1600480929">
    <title>Donnet , Rivoirard , Rousseau : Nonparametric Bayesian estimation for multivariate Hawkes processes</title>
    <dc:date>2020-11-18T22:50:20+00:00</dc:date>
    <link>https://projecteuclid.org/euclid.aos/1600480929</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["This paper studies nonparametric estimation of parameters of multivariate Hawkes processes. We consider the Bayesian setting and derive posterior concentration rates. First, rates are derived for 𝕃1L1-metrics for stochastic intensities of the Hawkes process. We then deduce rates for the 𝕃1L1-norm of interactions functions of the process. Our results are exemplified by using priors based on piecewise constant functions, with regular or random partitions and priors based on mixtures of Betas distributions. We also present a simulation study to illustrate our results and to study empirically the inference on functional connectivity graphs of neurons"]]></description>
<dc:subject>to:NB point_processes functional_connectivity bayesian_consistency nonparametrics statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:4b84047a5538/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:point_processes"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:functional_connectivity"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:bayesian_consistency"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/euclid.aos/1594972839">
    <title>Cannings , Berrett , Samworth : Local nearest neighbour classification with applications to semi-supervised learning</title>
    <dc:date>2020-11-18T22:44:02+00:00</dc:date>
    <link>https://projecteuclid.org/euclid.aos/1594972839</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We derive a new asymptotic expansion for the global excess risk of a local-kk-nearest neighbour classifier, where the choice of kk may depend upon the test point. This expansion elucidates conditions under which the dominant contribution to the excess risk comes from the decision boundary of the optimal Bayes classifier, but we also show that if these conditions are not satisfied, then the dominant contribution may arise from the tails of the marginal distribution of the features. Moreover, we prove that, provided the dd-dimensional marginal distribution of the features has a finite ρρth moment for some ρ>4ρ>4 (as well as other regularity conditions), a local choice of kk can yield a rate of convergence of the excess risk of O(n−4/(d+4))O(n−4/(d+4)), where nn is the sample size, whereas for the standard kk-nearest neighbour classifier, our theory would require d≥5d≥5 and ρ>4d/(d−4)ρ>4d/(d−4) finite moments to achieve this rate. These results motivate a new kk-nearest neighbour classifier for semi-supervised learning problems, where the unlabelled data are used to obtain an estimate of the marginal feature density, and fewer neighbours are used for classification when this density estimate is small. Our worst-case rates are complemented by a minimax lower bound, which reveals that the local, semi-supervised kk-nearest neighbour classifier attains the minimax optimal rate over our classes for the excess risk, up to a subpolynomial factor in nn. These theoretical improvements over the standard kk-nearest neighbour classifier are also illustrated through a simulation study."]]></description>
<dc:subject>classifiers nearest_neighbors statistics nonparametrics samworth.richard_j. in_NB</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:357925370f16/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:classifiers"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nearest_neighbors"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:samworth.richard_j."/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/euclid.aos/1594972822">
    <title>Nickl , Ray : Nonparametric statistical inference for drift vector fields of multi-dimensional diffusions</title>
    <dc:date>2020-11-18T21:47:49+00:00</dc:date>
    <link>https://projecteuclid.org/euclid.aos/1594972822</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["The problem of determining a periodic Lipschitz vector field b=(b1,…,bd)b=(b1,…,bd) from an observed trajectory of the solution (Xt:0≤t≤T)(Xt:0≤t≤T) of the multi-dimensional stochastic differential equation
dXt=b(Xt)dt+dWt,t≥0,
dXt=b(Xt)dt+dWt,t≥0,
where WtWt is a standard dd-dimensional Brownian motion, is considered. Convergence rates of a penalised least squares estimator, which equals the maximum a posteriori (MAP) estimate corresponding to a high-dimensional Gaussian product prior, are derived. These results are deduced from corresponding contraction rates for the associated posterior distributions. The rates obtained are optimal up to log-factors in L2L2-loss in any dimension, and also for supremum norm loss when d≤4d≤4. Further, when d≤3d≤3, nonparametric Bernstein–von Mises theorems are proved for the posterior distributions of bb. From this, we deduce functional central limit theorems for the implied estimators of the invariant measure μbμb. The limiting Gaussian process distributions have a covariance structure that is asymptotically optimal from an information-theoretic point of view."]]></description>
<dc:subject>to:NB stochastic_differential_equations nonparametrics statistical_inference_for_stochastic_processes statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:c8f20a18002d/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:stochastic_differential_equations"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistical_inference_for_stochastic_processes"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/euclid.aos/1594972820">
    <title>Liang , Rakhlin : Just interpolate: Kernel “Ridgeless” regression can generalize</title>
    <dc:date>2020-11-18T21:46:27+00:00</dc:date>
    <link>https://projecteuclid.org/euclid.aos/1594972820</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["In the absence of explicit regularization, Kernel “Ridgeless” Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for minimum-norm interpolated solutions which is due to a combination of high dimensionality of the input data, curvature of the kernel function and favorable geometric properties of the data such as an eigenvalue decay of the empirical covariance and kernel matrices. In addition to deriving a data-dependent upper bound on the out-of-sample error, we present experimental evidence suggesting that the phenomenon occurs in the MNIST dataset."]]></description>
<dc:subject>kernel_methods regression nonparametrics learning_theory statistics rakhlin.alexander to_teach:childs_garden_of_statistical_learning_theory in_NB interpolation_aka_memorizing_the_training_data</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:5e5a05c937da/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kernel_methods"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:rakhlin.alexander"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_teach:childs_garden_of_statistical_learning_theory"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:interpolation_aka_memorizing_the_training_data"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/2011.07275">
    <title>[2011.07275] Inference Functions for Semiparametric Models</title>
    <dc:date>2020-11-18T17:21:55+00:00</dc:date>
    <link>https://arxiv.org/abs/2011.07275</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["The paper discusses inference techniques for semiparametric models based on suitable versions of inference functions. The text contains two parts. In the first part, we review the optimality theory for non-parametric models based on the notions of path differentiability and statistical functional differentiability. Those notions are adapted to the context of semiparametric models by applying the inference theory of statistical functionals to the functional that associates the value of the interest parameter to the corresponding probability measure. The second part of the paper discusses the theory of inference functions for semiparametric models. We define a class of regular inference functions, and provide two equivalent characterisations of those inference functions: One adapted from the classic theory of inference functions for parametric models, and one motivated by differential geometric considerations concerning the statistical model. Those characterisations yield an optimality theory for estimation under semiparametric models. We present a necessary and sufficient condition for the coincidence of the bound for the concentration of estimators based on inference functions and the semiparametric Cramèr-Rao bound. Projecting the score function for the parameter of interest on specially designed spaces of functions, we obtain optimal inference functions. Considering estimation when a sufficient statistic is present, we provide an alternative justification for the conditioning principle in a context of semiparametric models. The article closes with a characterisation of when the semiparametric Cramèr-Rao bound is attained by estimators derived from regular inference functions."]]></description>
<dc:subject>to:NB nonparametrics statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:4ff47a06a003/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/euclid.ejs/1597975224">
    <title>Liu , Shang , Cheng : Nonparametric distributed learning under general designs</title>
    <dc:date>2020-11-16T16:22:30+00:00</dc:date>
    <link>https://projecteuclid.org/euclid.ejs/1597975224</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["This paper focuses on the distributed learning in nonparametric regression framework. With sufficient computational resources, the efficiency of distributed algorithms improves as the number of machines increases. We aim to analyze how the number of machines affects statistical optimality. We establish an upper bound for the number of machines to achieve statistical minimax in two settings: nonparametric estimation and hypothesis testing. Our framework is general compared with existing work. We build a unified frame in distributed inference for various regression problems, including thin-plate splines and additive regression under random design: univariate, multivariate, and diverging-dimensional designs. The main tool to achieve this goal is a tight bound of an empirical process by introducing the Green function for equivalent kernels. Thorough numerical studies back theoretical findings."]]></description>
<dc:subject>to:NB statistics distributed_systems regression nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:bf1d42c85277/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:distributed_systems"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/euclid.ejs/1576573369">
    <title>Kim , Lee , Lei : Global and local two-sample tests via regression</title>
    <dc:date>2020-11-16T16:11:48+00:00</dc:date>
    <link>https://projecteuclid.org/euclid.ejs/1576573369</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Two-sample testing is a fundamental problem in statistics. Despite its long history, there has been renewed interest in this problem with the advent of high-dimensional and complex data. Specifically, in the machine learning literature, there have been recent methodological developments such as classification accuracy tests. The goal of this work is to present a regression approach to comparing multivariate distributions of complex data. Depending on the chosen regression model, our framework can efficiently handle different types of variables and various structures in the data, with competitive power under many practical scenarios. Whereas previous work has been largely limited to global tests which conceal much of the local information, our approach naturally leads to a local two-sample testing framework in which we identify local differences between multivariate distributions with statistical confidence. We demonstrate the efficacy of our approach both theoretically and empirically, under some well-known parametric and nonparametric regression methods. Our proposed methods are applied to simulated data as well as a challenging astronomy data set to assess their practical usefulness."]]></description>
<dc:subject>to:NB two-sample_tests nonparametrics high-dimensional_statistics regression kith_and_kin lee.ann_b. lei.jing heard_the_talk</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:d95b8656b5cd/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:two-sample_tests"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:high-dimensional_statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:kith_and_kin"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:lee.ann_b."/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:lei.jing"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:heard_the_talk"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://ieeexplore.ieee.org/document/9136791">
    <title>&lt;italic&gt;k&lt;/italic&gt;-Vectors: An Alternating Minimization Algorithm for Learning Regression Functions - IEEE Journals &amp; Magazine</title>
    <dc:date>2020-11-16T16:08:36+00:00</dc:date>
    <link>https://ieeexplore.ieee.org/document/9136791</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["The k -vectors algorithm for learning regression functions proposed here is akin to the well-known k -means algorithm. Both algorithms partition the feature space, but unlike the k -means algorithm, the k -vectors algorithm aims to reconstruct the response rather than the feature. The partitioning rule of the algorithm is based on maximizing the correlation (inner product) of the feature vector with a set of k vectors, and generates polyhedral cells, similar to the ones generated by the nearest-neighbor rule of the k -means algorithm. Similarly to k -means, the learning algorithm alternates between two types of steps. In the first type of steps, k labels are determined via a centroid-type rule (in the response space), which uses a surrogate hinge-type loss function to the mean squared error loss function. In the second type of steps, the k vectors which determine the partition are updated according to a multiclass classification rule, in the spirit of support vector machines. It is proved that both steps of the algorithm only require solving convex optimization problems, and that the algorithm is empirically consistent - as the length of the training sequence increases to infinity, fixed-points of the empirical version of the algorithm tend to fixed points of the population version of the algorithm. Learnability of the predictor class posit by the algorithm is also established."]]></description>
<dc:subject>to:NB clustering regression nonparametrics statistics k-means nearest_neighbors to_read</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:fd9acdc3e960/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:clustering"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:k-means"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nearest_neighbors"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1910.00943">
    <title>[1910.00943] A note on the consistency of the random forest algorithm</title>
    <dc:date>2020-01-12T23:01:47+00:00</dc:date>
    <link>https://arxiv.org/abs/1910.00943</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Examples are given of data-generating models for which some versions of the random forest algorithm may fail to be consistent, or at least may be extremely slow to converge to the optimal predictor. The evidence provided for these properties is based on partly intuitive and partly rigorous arguments and on numerical experiments."]]></description>
<dc:subject>to:NB random_forests regression statistics nonparametrics to_teach:data-mining color_me_skeptical</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:be94315284cc/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:random_forests"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_teach:data-mining"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:color_me_skeptical"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1908.03606">
    <title>[1908.03606] Goodness-of-fit testing in high-dimensional generalized linear models</title>
    <dc:date>2020-01-12T22:26:07+00:00</dc:date>
    <link>https://arxiv.org/abs/1908.03606</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We propose a family of tests to assess the goodness-of-fit of a high-dimensional generalized linear model. Our framework is flexible and may be used to construct an omnibus test or directed against testing specific non-linearities and interaction effects, or for testing the significance of groups of variables. The methodology is based on extracting left-over signal in the residuals from an initial fit of a generalized linear model. This can be achieved by predicting this signal from the residuals using modern flexible regression or machine learning methods such as random forests or boosted trees. Under the null hypothesis that the generalized linear model is correct, no signal is left in the residuals and our test statistic has a Gaussian limiting distribution, translating to asymptotic control of type I error. Under a local alternative, we establish a guarantee on the power of the test. We illustrate the effectiveness of the methodology on simulated and real data examples by testing goodness-of-fit in logistic regression models. Software implementing the methodology is available in the R package `GRPtests'."]]></description>
<dc:subject>to:NB goodness-of-fit regression statistics linear_regression nonparametrics buhlmann.peter to_read re:ADAfaEPoV misspecification samworth.richard_j.</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:1d17667d900b/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:goodness-of-fit"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:linear_regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:buhlmann.peter"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:re:ADAfaEPoV"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:misspecification"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:samworth.richard_j."/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/euclid.ejs/1574240425">
    <title>Kleijn , Zhao : Criteria for posterior consistency and convergence at a rate</title>
    <dc:date>2019-12-01T22:58:13+00:00</dc:date>
    <link>https://projecteuclid.org/euclid.ejs/1574240425</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Frequentist conditions for asymptotic consistency of Bayesian procedures with i.i.d. data focus on lower bounds for prior mass in Kullback-Leibler neighbourhoods of the data distribution. The goal of this paper is to investigate the flexibility in these criteria. We derive a versatile new posterior consistency theorem, which is used to consider Kullback-Leibler consistency and indicate when it is sufficient to have a prior that charges metric balls instead of KL-neighbourhoods. We generalize our proposal to sieved models with Barron’s negligible prior mass condition and to separable models with variations on Walker’s condition. Results are also applied in semi-parametric consistency: support boundary estimation is considered explicitly and consistency is proved in a model for which Kullback-Leibler priors do not exist. As a further demonstration of applicability, we consider metric consistency at a rate: under a mild integrability condition, the second-order Ghosal-Ghosh-van der Vaart prior mass condition can be relaxed to a lower bound for ordinary KL-neighbourhoods. The posterior rate is derived in a parametric model for heavy-tailed distributions in which the Ghosal-Ghosh-van der Vaart condition cannot be satisfied by any prior."]]></description>
<dc:subject>to:NB statistics bayesian_consistency nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:dba2c5a15af5/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:bayesian_consistency"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1807.05748">
    <title>[1807.05748] Learning Stochastic Differential Equations With Gaussian Processes Without Gradient Matching</title>
    <dc:date>2019-11-10T21:46:11+00:00</dc:date>
    <link>https://arxiv.org/abs/1807.05748</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We introduce a novel paradigm for learning non-parametric drift and diffusion functions for stochastic differential equation (SDE). The proposed model learns to simulate path distributions that match observations with non-uniform time increments and arbitrary sparseness, which is in contrast with gradient matching that does not optimize simulated responses. We formulate sensitivity equations for learning and demonstrate that our general stochastic distribution optimisation leads to robust and efficient learning of SDE systems."]]></description>
<dc:subject>to:NB stochastic_differential_equations statistical_inference_for_stochastic_processes gaussian_processes statistics nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:5db7dca15d4e/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:stochastic_differential_equations"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistical_inference_for_stochastic_processes"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:gaussian_processes"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://projecteuclid.org/euclid.aos/1572487393">
    <title>Veitch , Roy : Sampling and estimation for (sparse) exchangeable graphs</title>
    <dc:date>2019-11-01T00:50:42+00:00</dc:date>
    <link>https://projecteuclid.org/euclid.aos/1572487393</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["Sparse exchangeable graphs on ℝ+R+, and the associated graphex framework for sparse graphs, generalize exchangeable graphs on ℕN, and the associated graphon framework for dense graphs. We develop the graphex framework as a tool for statistical network analysis by identifying the sampling scheme that is naturally associated with the models of the framework, formalizing two natural notions of consistent estimation of the parameter (the graphex) underlying these models, and identifying general consistent estimators in each case. The sampling scheme is a modification of independent vertex sampling that throws away vertices that are isolated in the sampled subgraph. The estimators are variants of the empirical graphon estimator, which is known to be a consistent estimator for the distribution of dense exchangeable graphs; both can be understood as graph analogues to the empirical distribution in the i.i.d. sequence setting. Our results may be viewed as a generalization of consistent estimation via the empirical graphon from the dense graph regime to also include sparse graphs."]]></description>
<dc:subject>to:NB graph_limits network_data_analysis graphons nonparametrics statistics veitch.victor to_teach:graphons</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:16e640f46676/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:graph_limits"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:network_data_analysis"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:graphons"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:veitch.victor"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_teach:graphons"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1907.02306">
    <title>[1907.02306] Consistent Regression using Data-Dependent Coverings</title>
    <dc:date>2019-10-29T14:34:19+00:00</dc:date>
    <link>https://arxiv.org/abs/1907.02306</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["In this paper, we introduce a novel method to generate interpretable regression function estimators. The idea is based on called data-dependent coverings. The aim is to extract from the data a covering of the feature space instead of a partition. The estimator predicts the empirical conditional expectation over the cells of the partitions generated from the coverings. Thus, such estimator has the same form as those issued from data-dependent partitioning algorithms. We give sufficient conditions to ensure the consistency, avoiding the sufficient condition of shrinkage of the cells that appears in the former literature. Doing so, we reduce the number of covering elements. We show that such coverings are interpretable and each element of the covering is tagged as significant or insignificant. The proof of the consistency is based on a control of the error of the empirical estimation of conditional expectations which is interesting on its own."]]></description>
<dc:subject>to:NB statistics regression nonparametrics to_read</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:37c6b7b378f3/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to_read"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1603.07632">
    <title>[1603.07632] Statistical inference in sparse high-dimensional additive models</title>
    <dc:date>2019-10-22T13:49:54+00:00</dc:date>
    <link>https://arxiv.org/abs/1603.07632</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["In this paper we discuss the estimation of a nonparametric component f1 of a nonparametric additive model Y=f1(X1)+...+fq(Xq)+ϵ. We allow the number q of additive components to grow to infinity and we make sparsity assumptions about the number of nonzero additive components. We compare this estimation problem with that of estimating f1 in the oracle model Z=f1(X1)+ϵ, for which the additive components f2,…,fq are known. We construct a two-step presmoothing-and-resmoothing estimator of f1 and state finite-sample bounds for the difference between our estimator and some smoothing estimators f̂ (oracle)1 in the oracle model. In an asymptotic setting these bounds can be used to show asymptotic equivalence of our estimator and the oracle estimators; the paper thus shows that, asymptotically, under strong enough sparsity conditions, knowledge of f2,…,fq has no effect on estimation accuracy. Our first step is to estimate f1 with an undersmoothed estimator based on near-orthogonal projections with a group Lasso bias correction. We then construct pseudo responses Ŷ  by evaluating a debiased modification of our undersmoothed estimator of f1 at the design points. In the second step the smoothing method of the oracle estimator f̂ (oracle)1 is applied to a nonparametric regression problem with responses Ŷ  and covariates X1. Our mathematical exposition centers primarily on establishing properties of the presmoothing estimator. We present simulation results demonstrating close-to-oracle performance of our estimator in practical applications."

--- ETA: Journal version, https://doi.org/10.1214/20-AOS2011]]></description>
<dc:subject>additive_models statistics regression nonparametrics sparsity in_NB</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:254865861d99/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:additive_models"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:regression"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:sparsity"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:in_NB"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1910.08477">
    <title>[1910.08477] Density estimation on an unknown submanifold</title>
    <dc:date>2019-10-21T15:51:21+00:00</dc:date>
    <link>https://arxiv.org/abs/1910.08477</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["We investigate density estimation from a n-sample in the Euclidean space ℝD, when the data is supported by an unknown submanifold M of possibly unknown dimension d<D under a reach condition. We study nonparametric kernel methods for pointwise and integrated loss, with data-driven bandwidths that incorporate some learning of the geometry via a local dimension estimator. When f has Hölder smoothness β and M has regularity α in a sense to be defined, our estimator achieves the rate n−α∧β/(2α∧β+d) and does not depend on the ambient dimension D and is asymptotically minimax for α≥β. Following Lepski's principle, a bandwidth selection rule is shown to achieve smoothness adaptation. We also investigate the case α≤β: by estimating in some sense the underlying geometry of M, we establish in dimension d=1 that the minimax rate is n−β/(2β+1) proving in particular that it does not depend on the regularity of M. Finally, a numerical implementation is conducted on some case studies in order to confirm the practical feasibility of our estimators."]]></description>
<dc:subject>to:NB density_estimation statistics_on_manifolds statistics nonparametrics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:5cf29a2b288f/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:density_estimation"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics_on_manifolds"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
</rdf:Bag></taxo:topics>
</item>
<item rdf:about="https://arxiv.org/abs/1910.06028">
    <title>[1910.06028] Accuracy of Gaussian approximation in nonparametric Bernstein -- von Mises Theorem</title>
    <dc:date>2019-10-16T14:07:23+00:00</dc:date>
    <link>https://arxiv.org/abs/1910.06028</link>
    <dc:creator>cshalizi</dc:creator><description><![CDATA["The prominent Bernstein -- von Mises (BvM) result claims that the posterior distribution after centering by the efficient estimator and standardizing by the square root of the total Fisher information is nearly standard normal. In particular, the prior completely washes out from the asymptotic posterior distribution. This fact is fundamental and justifies the Bayes approach from the frequentist viewpoint. In the nonparametric setup the situation changes dramatically and the impact of prior becomes essential even for the contraction of the posterior; see~\cite{vdV2008}, \cite{Bo2011}, \cite{CaNi2013,CaNi2014} for different models like Gaussian regression or i.i.d. model in different weak topologies. This paper offers another non-asymptotic approach to studying the behavior of the posterior for a special but rather popular and useful class of statistical models and for Gaussian priors. First we derive tight finite sample bounds on posterior contraction in terms of the so called effective dimension of the parameter space. Our main results describe the accuracy of Gaussian approximation of the posterior. In particular, we show that restricting to the class of all centrally symmetric credible sets around pMLE allows to get Gaussian approximation up to order \( n^{-1} \). We also show that the posterior distribution mimics well the distribution of the penalized maximum likelihood estimator (pMLE) and reduce the question of reliability of credible sets to consistency of the pMLE-based confidence sets. The obtained results are specified for nonparametric log-density estimation and generalized regression."]]></description>
<dc:subject>to:NB bayesian_consistency nonparametrics statistics</dc:subject>
<dc:source>https://pinboard.in/</dc:source>
<dc:identifier>https://pinboard.in/u:cshalizi/b:dca1f764b77f/</dc:identifier>
<taxo:topics><rdf:Bag>	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:to:NB"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:bayesian_consistency"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:nonparametrics"/>
	<rdf:li rdf:resource="https://pinboard.in/u:cshalizi/t:statistics"/>
</rdf:Bag></taxo:topics>
</item>
</rdf:RDF>