Matthew Avery

Applied Statistician & Data Scientist

Specializations

Predictive modeling, uncertainty estimation, mixed models, national defense, functional data, UAVs, test design

Education

North Carolina State University - Raleigh, NC

2012 PhD (Statistics) Dissertation: New Techniques for Functional Data Analysis: Model Selection, Classification, and Nonparametric Regression.

2009 Master’s (Statistics)

New College of Florida - Sarasota, FL

2006 Bachelor of Arts (Economics) Thesis: The Welfare Effects of Direct-to-Consumer Advertising in Prescription Drug Markets: A Theoretical Model and Cost-Benefit Analysis

Professional Experience

Institute for Defense Analyses, Alexandria, VA

2020 - Present Co-Chair, Data Governance Working Group

Founded and secured funding for Working Group leading IDA’s efforts to draft and implement Data Management policies.

2017 – Present Task Leader, Tactical Aerial Reconnaissance Systems (TARS)

Manage $300k line of effort providing analysis on unmanned aircraft systems. Coordinate work of nine IDA researchers to provide the Office of the Director, Operational Test and Evaluation input on six Programs of Record. Tasks include budget planning, organizing coverage plans for test events, and quality control of team products.

2018 – 2019 Embedded Analyst, Office of Cost Analysis and Program Evaluation (CAPE)

Provide analytical support to OSD(CAPE). Perform original data analysis and organize ad hoc teams of IDA researchers to address special requests from CAPE staff.

2012 – 2016 Research Staff Member

Provided analytical support to the Office of the Director, Operational Test and Evaluation on over twenty DoD Programs of Record. Products included planning and design of operational tests, observing test events, data analysis, and report writing.

Virginia Tech (National Capital Region), Arlington, VA

2020 - Present Adjunct Professor, Data Analytics and Applied Sciences

Instructor for Statistics in Research course covering sampling, one- and two-sample t-tests and one- and two-sample tests for proportions, confidence intervals for sample means and proportions, regression, ANOVA, and nonparametric tests.

North Carolina State University, Raleigh NC

Summer 2019 Summer Lecturer

Managed online semester-long course on Experimental Statistics, covering regression, ANOVA, generalized linear models, mixed models, and split plots.

2012 – 2016 Teaching Assistant

Sole instructor for introductory and second-semester statistics courses for non-majors (five total semesters). Prepared and presented lectures, produced in-class activities, provided one-on-one instruction to students, prepared and graded exams.

Merscom/Playdom, Chapel Hill NC

2009-2010 Business Analytics Specialist (Internship)

Provided direct analytical support to Merscom chief executives (and Chapel Hill office leadership post-acquisition by Playdom). Tracked game play metrics and provided daily reports to leadership monitoring trends including re-visit rates and uptake.

Teaching

North Carolina State University

2012-2011 ST 312: Introduction to Statistics II Material includes multiple linear regression, one- and two-way ANOVA, multiple comparisons, and other topics. Responsibilities included preparing lectures, designing in-class activities, and writing exams.

2008-2011 ST 311: Introduction to Statistics I Material includes hypothesis testing for means and proportions, confidence intervals, sampling, introduction to regression and other topics. Responsibilities included preparing lectures, designing in-class activities, and writing midterm exams.

Short Courses and Tutorials

Multiple Design of Experiments DOT&E Action Officer Training

Multiple Reliability Test Planning DOT&E Action Officer Training

Multiple instructors help to build and present sections of these full-day courses, which are delivered once or twice annually to Action Officers working for the Office of the Director, Operational Test and Evaluation. Author and presenter responsibilities vary from year to year. Relevant sections include Experimental Designs; Data Analysis Overview; Reliability, Availability, and Maintainability (RAM) Requirements; and Analysis of RAM Data.

April 2016 Bootstrapping 101 Science of Test Workshop

Half-day tutorial covering populations and sampling, sampling distributions, statistical inference, resampling, the “bootstrap world”, confidence intervals for simple and complex statistics, hypothesis testing, and further extensions.

July 2014 Maximum Likelihood Estimation Institute for Defense Analyses

Half-day course covering estimation and properties of MLEs, multi-parameter MLEs, censoring and transformations, hypothesis testing, confidence intervals, Fisher Information, and the delta method.

Fellowships and Awards

2014 Runner-up Outstanding JSM Presentation Award American Statistical Association Section on Statistics in the Physical and Engineering Sciences

2014 Best Presentation Award International Test and Evaluation Association

2010-2011 Graduate Fellow Statistical and Applied Mathematical Sciences Institute

Selected Presentations

External

January 2018 ciTools: An R Package to Ease Model Inference rstudioconf::2018

August 2018 ciTools: Quantifying Uncertainty for Statistical Models Joint Statistical Meetings

April 2017 Session Chair, “Data Synthesis and Visualization with Big Data” Science of Test Workshop

August 2017 Uncertainty Quantification for Mixed Models Joint Statistical Meetings

August 2016 Applications of Modern Statistical Techniques to Operational Testing Joint Statistical Meetings

September 2015 Ordinal Response Variables Observed Continuously Over Time European Network for Business and Industrial Statistics

January 2015 Reliability Survey of Military Acquisition Systems Reliability and Maintainability Symposium

December 2014 Empirical Signal to Noise Ratios for Operational Tests Conference on Applied Statistics and Defense

October 2014 2013 Reliability Survey 31st International Test and Evaluation Symposium

August 2014 Session Chair, “Statistical Methods for Advancing Defense and National Security” Joint Statistical Meetings

August 2013 Experimental Design Challenges in Operational Testing: Recent Case Studies Joint Statistical Meetings

Internal (IDA)

March 2018 Highlights from rstudioconf::2018

November 2017 Thinking About Data for Operational Test and Evaluation

October 2017 Simple Results from Complex Data: Analyzing IDECM Block 4 Streamfire Data

May 2017 Reproducible Research for OED

February 2017 Highlights from rstudioconf::2017

January 2017 What Selecting “Hard-to-Change” Really Means: An Introduction to Split-plot Designs

October 2016 2015 Reliability Assessment

June 2016 First Steps into the Bootstrap World

October 2015 2014 Reliability Assessment

August 2014 Empirical Signal-to-Noise Ratios for Operational Test Data

August 2014 2013 Reliability Assessment

Selected Publications

Peer Reviewed Journals

Haman, John T., Avery, Matthew R. and Wojton, Heather M. 2019. “The Purpose of Mixed-Effects Models in Test and Evaluation.” ITEA Journal December, 2019.

Freeman L. J., Johnson, T., Avery, M., Lillard, V., Clutter, J. “Testing Defense Systems.” Analytic Methods in Systems and Software Testing. © 2018, pages 439-487 John Wiley & Sons, Ltd. Editors: Kenett, R., Ruggeri, F., Faltin, F.

Avery, M., Avery, K., Freeman, L.J. “Statistical Methods for Defense Testing.” Wiley Statistics Reference Online (2018).

Avery, M., Orndorff, M., Robinson, T., & Freeman, L.J. “Regularization for Continuously Observed Ordinal Response Variables with Piecewise‐constant Functional Covariates.” Quality and Reliability Engineering International (2016) 32(6).

J. L. Bell, M. R. Avery and M. C. Wells, “Reliability survey of military acquisition systems,” 2015 Annual Reliability and Maintainability Symposium (RAMS), Palm Harbor, FL, 2015, pp. 1-6.

Haslauer, C. M., Avery, M. R., Pourdeyhimi, B., & Loboa, E. G. (2015). “Translating Textiles to Tissue Engineering: Creation and Evaluation of Microporous, Biocompatible, Degradable Scaffolds Using Industry Relevant Manufacturing Approaches and Human Adipose Derived Stem Cells.” Journal of Biomedical Materials Research. Part B, Applied Biomaterials, 103(5), 1050–1058.

Avery M., Wu Y., Zhang H., Zhang J. “RKHS-based functional nonparametric regression for sparse and irregular longitudinal data.” Canadian Journal of Statistics, (2014) 42 (2) , pp. 204-216.

Avery, M. & Banks, H. & Basu, K. & Cheng, Y. & Eager, E. & Khasawinah, S. & Potter, L. & Rehm, K. (2012). “Experimental design and inverse problems in plant biological modeling.” Journal of Inverse and Ill-Posed Problems. 20. 169–191.

Peer Reviewed Technical Reports

Wojton, Heather M., Avery, Kelly M., Avery, Matthew R., Medlin, Rebecca M. and Lane, Stephanie T. 2019. “Test Design and Reporting AO Course”. IDA Document NS D-9227.

Lillard, Vincent A., Avery, Matthew R., Koretsky, Geoffrey M., Slawik, Alexander J., Ouellette, Stephen M., Thorne, James D., de Naray, Rachel K., Adelizzi, Eric A. and Goeller, Lawrence N. 2018. “Satellite Affordability in LEO (SAL)”. IDA Document NS D-9056.

Pechacek, Julie A., Avery, Matthew R., Bishop, James M., Picucci, P. M., Romana, Amrit K. and Saizan, Alexandra M. “Design for a Controlled Trial of an Alcohol Market Intervention Intended to Reduce Harmful Behaviors in the Army and Air Force”. (2018) IDA Paper P-9185.

Brooks, Peter S., Avery, Matthew R., Avery, Kelly M., Moses, Franklin L. and Thorne, James D. “(U) Effectiveness of ISR Assets Against Hard Targets”. (2018) IDA Document NS D-9145, -9146, and -9147

Thomas, Dean, Goodman, Allison L., Avery, Matthew R., Bell, Jonathan L., Dickinson, Rebecca M., Hueckstaedt, Robert M., Peek, Douglas A., Roberts, Max W. and Freeman, Laura J. “Assessment of Reliability in DoD Acquisition Programs”. (2018) IDA Document NS D-8997.

Schulman, Eric R., Avery, Matthew R., Corbett, Brooke M., Porter, Daniel J., Wall, Kenneth E., Wojton, Heather M. and Luhman, Michael L. “(U) Independent Evaluation of the Patriot Post-Deployment Build-8 Initial Operational Test and Evaluation”. (2018) IDA Document D-8665.

Freeman, Laura J. and Avery, Matthew R. “Prediction Uncertainty for Autocorrelated Lognormal Data with Random Effects”. (2017) IDA Document NS D-8629.

Peterson, Robert G., Schulman, Eric R., Avery, Matthew R., Schlick, Conor P. and Filkins, Peter C. “(U) Independent Evaluation of the Joint Standoff Weapon C-1 Follow-on Operational Test and Evaluation”. (2017). IDA Paper P-8369.

Shaw, Scott E., Avery, Matthew R., Crabtree, Brent A., Grandinetti, Saul A. and Lambrecht, Kristen L. “RQ-7BV2 Shadow Tactical Unmanned Aircraft System Follow-on Operational Test and Evaluation, and Operational Assessment of the One System Remote Video Terminal”. (2015). IDA Document D-5373.

Shaw, Scott E., Avery, Matthew R., Brathwaite, Joy D., Eveker, Kevin M. and Lambrecht, Kristen L. “(U) Independent Assessment of the RQ-21A Blackjack Small Tactical Unmanned Aircraft System - Volume II - Appendix C, (U) RQ-21A Blackjack Cybersecurity Assessment”. (2016). IDA Paper P-5270 (Volumes 1 and 2).

Carlson, Charles D., Satyapal, Shanti, Ratchford, Jasmine, Avery, Matthew R. and Kirpichevsky, Yevgeniy. “Test and Evaluation Concept for the Biometrics Enabling Capability (BEC) Increment 0 IOT&E”. (2014). IDA Document D-5078.

Shaw, Scott E. and Avery, Matthew R. “Land-Based Phase of the MQ-8B Weapons Quick Reaction Assessment”. (2013). IDA Document D-5051.

Thayer, Bradley D., Mathiasmeier, Kenneth J. and Avery, Matthew R. “Test and Evaluation Concept for the Next Generation Jammer (NGJ)”. (2014). IDA Document D-4972.

Avery, Matthew R. and Shaw, Scott E. “Assessment of RQ-21A Small Tactical Unmanned Aircraft System Performance During Operational Test Period B2”. (2013). IDA Document D-4904.

Memberships

International Test and Evaluation Association (ITEA)

American Statistical Association