Copyright 2023 ACM, Inc. Understanding black-box predictions via influence functions. Loss non-convex, quadratic loss . Understanding Black-box Predictions via Influence Functions The implicit and explicit regularization effects of dropout. Limitations of the empirical Fisher approximation for natural gradient descent. Understanding short-horizon bias in stochastic meta-optimization. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. Influence functions can of course also be used for data other than images, The final report is due April 7. The next figure shows the same but for a different model, DenseNet-100/12. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. multilayer perceptrons), you can use straight-up JAX so that you understand everything that's going on. In Artificial Intelligence and Statistics (AISTATS), pages 3382-3390, 2019. We'll mostly focus on minimax optimization, or zero-sum games. Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. An empirical model of large-batch training. Often we want to identify an influential group of training samples in a particular test prediction for a given machine learning model. Proc 34th Int Conf on Machine Learning, p.1885-1894. Debruyne, M., Hubert, M., and Suykens, J. the original paper linked here. Data poisoning attacks on factorization-based collaborative filtering. Assignments for the course include one problem set, a paper presentation, and a final project. $-hm`nrurh%\L(0j/hM4/AO*V8z=./hQ-X=g(0 /f83aIF'Mu2?ju]n|# =7$_--($+{=?bvzBU[.Q. This is a better choice if you want all the bells-and-whistles of a near-state-of-the-art model. Theano: A Python framework for fast computation of mathematical expressions. Please try again. Are you sure you want to create this branch? With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. Fast exact multiplication by the hessian. Understanding Black-box Predictions via Influence Functions - Github LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. The datasets for the experiments can also be found at the Codalab link. There are various full-featured deep learning frameworks built on top of JAX and designed to resemble other frameworks you might be familiar with, such as PyTorch or Keras. Gradient descent on neural networks typically occurs on the edge of stability. You can get the default config by calling ptif.get_default_config(). and Hessian-vector products. On the importance of initialization and momentum in deep learning, A mathematical theory of semantic development in deep neural networks. calculated. We see how to approximate the second-order updates using conjugate gradient or Kronecker-factored approximations. J. Cohen, S. Kaur, Y. Li, J. Requirements Installation Usage Background and Documentation config Misc parameters approximations to influence functions can still provide valuable information. In many cases, they have far more than enough parameters to memorize the data, so why do they generalize well? Understanding Black-box Predictions via Influence Functions - PMLR In this lecture, we consider the behavior of neural nets in the infinite width limit. Understanding Black-box Predictions via Influence Functions (2017) This In contrast with TensorFlow and PyTorch, JAX has a clean NumPy-like interface which makes it easy to use things like directional derivatives, higher-order derivatives, and differentiating through an optimization procedure. Disentangled graph convolutional networks. : , , , . Students are encouraged to attend class each week. ( , , ). on the final predictions is straight forward. Jaeckel, L. A. This is the case because grad_z has to be calculated twice, once for influences. ; Liang, Percy. This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: The datasets for the experiments can also be found at the Codalab link. Your search export query has expired. (b) 7 , 7 . In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1885--1894. lage2019evaluationI. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. Reconciling modern machine-learning practice and the classical bias-variance tradeoff. In this paper, we use influence functions a classic technique from robust statistics to trace a models prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. In. Imagenet classification with deep convolutional neural networks. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. Online delivery. arXiv preprint arXiv:1703.04730 (2017). SVM , . Explain and Predict, and then Predict Again | Proceedings of the 14th We are given training points z 1;:::;z n, where z i= (x i;y i) 2 XY . Fortunately, influence functions give us an efficient approximation. The ACM Digital Library is published by the Association for Computing Machinery. Biggio, B., Nelson, B., and Laskov, P. Poisoning attacks against support vector machines. A sign-up sheet will be distributed via email. , . Most importantnly however, s_test is only The meta-optimizer has to confront many of the same challenges we've been dealing with in this course, so we can apply the insights to reverse engineer the solutions it picks. The reference implementation can be found here: link. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks.See more on this video at https://www.microsoft.com/en-us/research/video/understanding-black-box-predictions-via-influence-functions/ Things get more complicated when there are multiple networks being trained simultaneously to different cost functions. In. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. Systems often become easier to analyze in the limit. Biggio, B., Nelson, B., and Laskov, P. Support vector machines under adversarial label noise. Fast convergence of natural gradient descent for overparameterized neural networks. [ICML] Understanding Black-box Predictions via Influence Functions Aggregated momentum: Stability through passive damping. Understanding Black-box Predictions via Influence Functions Background information ICML 2017 best paper Stanford Pang Wei Koh CourseraStanfordNIPS 2019influence function Percy Liang11Michael Jordan Abstract Understanding black-box predictions via influence functions Computing methodologies Machine learning Recommendations On second-order group influence functions for black-box predictions With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. This is a tentative schedule, which will likely change as the course goes on. Overwhelmed? Often we want to identify an influential group of training samples in a particular test prediction for a given We study the task of hardness amplification which transforms a hard function into a harder one. Delta-STN: Efficient bilevel optimization of neural networks using structured response Jacobians. Which optimization techniques are useful at which batch sizes? In, Mei, S. and Zhu, X. We'll consider the heavy ball method and why the Nesterov Accelerated Gradient can further speed up convergence. samples for each test data sample. We are preparing your search results for download We will inform you here when the file is ready. On the Accuracy of Influence Functions for Measuring - ResearchGate In, Martens, J. the first approximation in s_test and once to combine with the s_test Understanding Black-box Predictions via Influence Functions Proceedings of the 34th International Conference on Machine Learning . , loss , input space . Besides just getting your networks to train better, another important reason to study neural net training dynamics is that many of our modern architectures are themselves powerful enough to do optimization. Idea: use Influence Functions to observe the influence of the test samples from the training samples. S. L. Smith, B. Dherin, D. Barrett, and S. De. This isn't the sort of applied class that will give you a recipe for achieving state-of-the-art performance on ImageNet. influence function. Kansagara, D., Englander, H., Salanitro, A., Kagen, D., Theobald, C., Freeman, M., and Kripalani, S. Risk prediction models for hospital readmission: a systematic review. But keep in mind that some of the key concepts in this course, such as directional derivatives or Hessian-vector products, might not be so straightforward to use in some frameworks. Measuring and regularizing networks in function space. ICML 2017 Best Paper - How can we explain the predictions of a black-box model? Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., and Suh, J. Modeltracker: Redesigning performance analysis tools for machine learning. Pearlmutter, B. Class will be held synchronously online every week, including lectures and occasionally tutorials. It is individual work. grad_z on the other hand is only dependent on the training The first mode is called calc_img_wise, during which the two as long as you have a supervised learning problem. Here, we used CIFAR-10 as dataset. << Interacting with predictions: Visual inspection of black-box machine learning models. If the influence function is calculated for multiple We show that even on non-convex and non-differentiable models The deep bootstrap framework: Good online learners are good offline generalizers. When can we take advantage of parallelism to train neural nets? For toy functions and simple architectures (e.g. Validations 4. Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. Uses cases Roadmap 2 Reviving an "old technique" from Robust statistics: Influence function to trace a model's prediction through the learning algorithm and back to its training data, Thomas, W. and Cook, R. D. Assessing influence on predictions from generalized linear models. Second-Order Group Influence Functions for Black-Box Predictions The security of latent Dirichlet allocation. prediction outcome of the processed test samples. Kingma, D. and Ba, J. Adam: A method for stochastic optimization. Model-agnostic meta-learning for fast adaptation of deep networks. S. Arora, S. Du, W. Hu, Z. Li, and R. Wang. thereby identifying training points most responsible for a given prediction. To scale up influence functions to modern machine learning settings, Understanding Black-box Predictions via Influence Functions Understanding black-box predictions via influence functions. Then, it'll calculate all s_test values and save those to disk. For this class, we'll use Python and the JAX deep learning framework. (a) What is the effect of the training loss and H 1 ^ terms in I up,loss? % , mislabel . Understanding black-box predictions via influence functions. Often we want to identify an influential group of training samples in a particular test prediction. One would have expected this success to require overcoming significant obstacles that had been theorized to exist. Rather, the aim is to give you the conceptual tools you need to reason through the factors affecting training in any particular instance. Using machine teaching to identify optimal training-set attacks on machine learners. I am grateful to my supervisor Tasnim Azad Abir sir, for his . D. Maclaurin, D. Duvenaud, and R. P. Adams. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. reading both values from disk and calculating the influence base on them. Approach Consider a prediction problem from some input space X (e.g., images) to an output space Y(e.g., labels). A spherical analysis of Adam with batch normalization. Lage, E. Chen, J. How can we explain the predictions of a black-box model? While these topics had consumed much of the machine learning research community's attention when it came to simpler models, the attitude of the neural nets community was to train first and ask questions later. Agarwal, N., Bullins, B., and Hazan, E. Second order stochastic optimization in linear time. Time permitting, we'll also consider the limit of infinite depth. Understanding Black-box Predictions via Influence Functions (2017) 1. RelEx: A Model-Agnostic Relational Model Explainer How can we explain the predictions of a black-box model? Riemannian metrics for neural networks I: Feed-forward networks. This leads to an important optimization tool called the natural gradient. ( , ?) In Proceedings of the international conference on machine learning (ICML). The dict structure looks similiar to this: Harmful is a list of numbers, which are the IDs of the training data samples Acknowledgements The authors of the conference paper 'Understanding Black-box Predictions via Influence Functions' Pang Wei Koh et al. PDF Understanding Black-box Predictions via Influence Functions - GitHub Pages Therefore, if we bring in an idea from optimization, we need to think not just about whether it will minimize a cost function faster, but also whether it does it in a way that's conducive to generalization. This is "Understanding Black-box Predictions via Influence Functions --- Pang Wei Koh, Percy Liang" by TechTalksTV on Vimeo, the home for high quality Understanding Black-box Predictions via Influence Functions. Google Scholar Digital Library; Josua Krause, Adam Perer, and Kenney Ng. This alert has been successfully added and will be sent to: You will be notified whenever a record that you have chosen has been cited. Are you sure you want to create this branch? If Influence Functions are the Answer, Then What is the Question? the algorithm will then calculate the influence functions for all images by Components of inuence. However, in a lower Data-trained predictive models see widespread use, but for the most part they are used as black boxes which output a prediction or score. Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. . You signed in with another tab or window. Depending what you're trying to do, you have several options: You are welcome to use whatever language and framework you like for the final project. The infinitesimal jackknife. Dependencies: Numpy/Scipy/Scikit-learn/Pandas In, Metsis, V., Androutsopoulos, I., and Paliouras, G. Spam filtering with naive Bayes - which naive Bayes? CSC2541 Winter 2021 - Department of Computer Science, University of Toronto A classic result by Radford Neal showed that (using proper scaling) the distribution of functions of random neural nets approaches a Gaussian process. Applications - Understanding model behavior Inuence functions reveal insights about how models rely on and extrapolate from the training data. Tasha Nagamine, . C. Maddison, D. Paulin, Y.-W. Teh, B. O'Donoghue, and A. Doucet. /Filter /FlateDecode Understanding Black-box Predictions via Influence Functions ICML2017 3 (influence function) 4 In, Cadamuro, G., Gilad-Bachrach, R., and Zhu, X. Debugging machine learning models. Metrics give a local notion of distance on a manifold. , Hessian-vector . In many cases, the distance between two neural nets can be more profitably defined in terms of the distance between the functions they represent, rather than the distance between weight vectors. A unified analysis of extra-gradient and optimistic gradient methods for saddle point problems: Proximal point approach. Gradient-based Hyperparameter Optimization through Reversible Learning. Proceedings of Machine Learning Research | Proceedings of the 34th Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. non-convex non-differentialble . Stochastic gradient descent as approximate Bayesian inference. Thus, in the calc_img_wise mode, we throw away all grad_z On the accuracy of influence functions for measuring group effects. When testing for a single test image, you can then Simonyan, K., Vedaldi, A., and Zisserman, A. For details and examples, look here. For more details please see Linearization is one of our most important tools for understanding nonlinear systems. influence-instance. Understanding black-box predictions via influence functions We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. On Second-Order Group Influence Functions for Black-Box Predictions Insights from a noisy quadratic model. ICML 2017 best paperStanfordPang Wei KohCourseraStanfordNIPS 2019influence functionPercy Liang11Michael Jordan, , \hat{\theta}_{\epsilon, z} \stackrel{\text { def }}{=} \arg \min _{\theta \in \Theta} \frac{1}{n} \sum_{i=1}^{n} L\left(z_{i}, \theta\right)+\epsilon L(z, \theta), \left.\mathcal{I}_{\text {up, params }}(z) \stackrel{\text { def }}{=} \frac{d \hat{\theta}_{\epsilon, z}}{d \epsilon}\right|_{\epsilon=0}=-H_{\tilde{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta}), , loss, \begin{aligned} \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) &\left.\stackrel{\text { def }}{=} \frac{d L\left(z_{\text {test }}, \hat{\theta}_{\epsilon, z}\right)}{d \epsilon}\right|_{\epsilon=0} \\ &=\left.\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} \frac{d \hat{\theta}_{\epsilon, z}}{d \epsilon}\right|_{\epsilon=0} \\ &=-\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} H_{\hat{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta}) \end{aligned}, \varepsilon=-1/n , z=(x,y) \\ z_{\delta} \stackrel{\text { def }}{=}(x+\delta, y), \hat{\theta}_{\epsilon, z_{\delta},-z} \stackrel{\text { def }}{=}\arg \min _{\theta \in \Theta} \frac{1}{n} \sum_{i=1}^{n} L\left(z_{i}, \theta\right)+\epsilon L\left(z_{\delta}, \theta\right)-\epsilon L(z, \theta), \begin{aligned}\left.\frac{d \hat{\theta}_{\epsilon, z_{\delta},-z}}{d \epsilon}\right|_{\epsilon=0} &=\mathcal{I}_{\text {up params }}\left(z_{\delta}\right)-\mathcal{I}_{\text {up, params }}(z) \\ &=-H_{\hat{\theta}}^{-1}\left(\nabla_{\theta} L(z_{\delta}, \hat{\theta})-\nabla_{\theta} L(z, \hat{\theta})\right) \end{aligned}, \varepsilon \delta \deltaloss, \left.\frac{d \hat{\theta}_{\epsilon, z_{\delta},-z}}{d \epsilon}\right|_{\epsilon=0} \approx-H_{\hat{\theta}}^{-1}\left[\nabla_{x} \nabla_{\theta} L(z, \hat{\theta})\right] \delta, \hat{\theta}_{z_{i},-z}-\hat{\theta} \approx-\frac{1}{n} H_{\hat{\theta}}^{-1}\left[\nabla_{x} \nabla_{\theta} L(z, \hat{\theta})\right] \delta, \begin{aligned} \mathcal{I}_{\text {pert,loss }}\left(z, z_{\text {test }}\right)^{\top} &\left.\stackrel{\text { def }}{=} \nabla_{\delta} L\left(z_{\text {test }}, \hat{\theta}_{z_{\delta},-z}\right)^{\top}\right|_{\delta=0} \\ &=-\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z, \hat{\theta}) \end{aligned}, train lossH \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) , -y_{\text {test }} y \cdot \sigma\left(-y_{\text {test }} \theta^{\top} x_{\text {test }}\right) \cdot \sigma\left(-y \theta^{\top} x\right) \cdot x_{\text {test }}^{\top} H_{\hat{\theta}}^{-1} x, influence functiondebug training datatraining point \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) losstraining pointtraining point, Stochastic estimationHHHTFO(np)np, ImageNetdogfish900Inception v3SVM with RBF kernel, poisoning attackinfluence function59157%77%10590/591, attackRelated worktraining set attackadversarial example, influence functionbad case debug, labelinfluence function, \mathcal{I}_{\text {up,loss }}\left(z_{i}, z_{i}\right) , 10%labelinfluence functiontrain lossrandom, \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right), \mathcal{I}_{\text {up,loss }}\left(z_{i}, z_{i}\right), \mathcal{I}_{\text {pert,loss }}\left(z, z_{\text {test }}\right)^{\top}, H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z, \hat{\theta}), Less Is Better: Unweighted Data Subsampling via Influence Function, influence functionleave-one-out retraining, 0.86H, SVMhinge loss0.95, straightforwardbest paper, influence functionloss. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. We'll start off the class by analyzing a simple model for which the gradient descent dynamics can be determined exactly: linear regression. In. In. Pang Wei Koh and Percy Liang. Springenberg, J. T., Dosovitskiy, A., Brox, T., and Riedmiller, M. Striving for simplicity: The all convolutional net. The datasets for the experiments can also be found at the Codalab link. The canonical example in machine learning is hyperparameter optimization. Here, we plot I up,loss against variants that are missing these terms and show that they are necessary for picking up the truly inuential training points. I'll attempt to convey our best modern understanding, as incomplete as it may be. Alex Adam, Keiran Paster, and Jenny (Jingyi) Liu, 25% Colab notebook and paper presentation. Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks. Reference Understanding Black-box Predictions via Influence Functions The power of interpolation: Understanding the effectiveness of SGD in modern over-parameterized learning. (a) train loss, Hessian, train_loss + Hessian . In this paper, we use influence functions --- a classic technique from robust statistics --- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Cook, R. D. Assessment of local influence. Measuring the effects of data parallelism on neural network training.
Categories
