Matthew Avery

Applied Statistician & Data Scientist

Specializations

Predictive modeling, uncertainty estimation, mixed models, national defense, functional data, UAVs, test design

Education

North Carolina State University - Raleigh, NC

2012 PhD (Statistics) Dissertation: New Techniques for Functional Data Analysis: Model Selection, Classification, and Nonparametric Regression.

2009 Master’s (Statistics)

New College of Florida - Sarasota, FL

2006 Bachelor of Arts (Economics) Thesis: The Welfare Effects of Direct-to-Consumer Advertising in Prescription Drug Markets: A Theoretical Model and Cost-Benefit Analysis

Professional Experience

Institute for Defense Analyses, Alexandria, VA

2018 – Present Embedded Analyst, Office of Cost Analysis and Program Evaluation (CAPE)

Provide analytical support to OSD(CAPE). Perform original data analysis and organize ad hoc teams of IDA researchers to address special requests from CAPE staff.

2017 – Present Task Leader, Tactical Aerial Reconnaissance Systems (TARS)

Manage $300k line of effort providing analysis on unmanned aircraft systems. Coordinate work of nine IDA researchers to provide the Office of the Director, Operational Test and Evaluation input on six Programs of Record. Tasks include budget planning, organizing coverage plans for test events, and quality control of team products.

2012 – 2016 Research Staff Member

Provided analytical support to the Office of the Director, Operational Test and Evaluation on over twenty DoD Programs of Record. Products included planning and design of operational tests, observing test events, data analysis, and report writing.

North Carolina State University, Raleigh NC

2012 – 2016 Teaching Assistant

Sole instructor for introductory and second-semester statistics courses for non-majors (five total semesters). Prepared and presented lectures, produced in-class activities, provided one-on-one instruction to students, prepared and graded exams.

Merscom/Playdom, Chapel Hill NC

2009-2010 Business Analytics Specialist (Internship)

Provided direct analytical support to Merscom chief executives (and Chapel Hill office leadership post-acquisition by Playdom). Tracked game play metrics and provided daily reports to leadership monitoring trends including re-visit rates and uptake.

Teaching

North Carolina State University

2012-2011 ST 312: Introduction to Statistics II Material includes multiple linear regression, one- and two-way ANOVA, multiple comparisons, and other topics. Responsibilities included preparing lectures, designing in-class activities, and writing exams.

2008-2011 ST 311: Introduction to Statistics I Material includes hypothesis testing for means and proportions, confidence intervals, sampling, introduction to regression and other topics. Responsibilities included preparing lectures, designing in-class activities, and writing midterm exams.

Short Courses and Tutorials

Multiple Design of Experiments DOT&E Action Officer Training

Multiple Reliability Test Planning DOT&E Action Officer Training

Multiple instructors help to build and present sections of these full-day courses, which are delivered once or twice annually to Action Officers working for the Office of the Director, Operational Test and Evaluation. Author and presenter responsibilities vary from year to year. Relevant sections include Experimental Designs; Data Analysis Overview; Reliability, Availability, and Maintainability (RAM) Requirements; and Analysis of RAM Data.

April 2016 Bootstrapping 101 Science of Test Workshop

Half-day tutorial covering populations and sampling, sampling distributions, statistical inference, resampling, the “bootstrap world”, confidence intervals for simple and complex statistics, hypothesis testing, and further extensions.

July 2014 Maximum Likelihood Estimation Institute for Defense Analyses

Half-day course covering estimation and properties of MLEs, multi-parameter MLEs, censoring and transformations, hypothesis testing, confidence intervals, Fisher Information, and the delta method.

Fellowships and Awards

2014 Runner-up Outstanding JSM Presentation Award American Statistical Association Section on Statistics in the Physical and Engineering Sciences

2014 Best Presentation Award International Test and Evaluation Association

2010-2011 Graduate Fellow Statistical and Applied Mathematical Sciences Institute

Selected Presentations

External

January 2018 ciTools: An R Package to Ease Model Inference rstudioconf::2018

August 2018 ciTools: Quantifying Uncertainty for Statistical Models Joint Statistical Meetings

April 2017 Session Chair, “Data Synthesis and Visualization with Big Data” Science of Test Workshop

August 2017 Uncertainty Quantification for Mixed Models Joint Statistical Meetings

August 2016 Applications of Modern Statistical Techniques to Operational Testing Joint Statistical Meetings

September 2015 Ordinal Response Variables Observed Continuously Over Time European Network for Business and Industrial Statistics

January 2015 Reliability Survey of Military Acquisition Systems Reliability and Maintainability Symposium

December 2014 Empirical Signal to Noise Ratios for Operational Tests Conference on Applied Statistics and Defense

October 2014 2013 Reliability Survey 31st International Test and Evaluation Symposium

August 2014 Session Chair, “Statistical Methods for Advancing Defense and National Security” Joint Statistical Meetings

August 2013 Experimental Design Challenges in Operational Testing: Recent Case Studies Joint Statistical Meetings

Internal (IDA)

March 2018 Highlights from rstudioconf::2018

November 2017 Thinking About Data for Operational Test and Evaluation

October 2017 Simple Results from Complex Data: Analyzing IDECM Block 4 Streamfire Data

May 2017 Reproducible Research for OED

February 2017 Highlights from rstudioconf::2017

January 2017 What Selecting “Hard-to-Change” Really Means: An Introduction to Split-plot Designs

October 2016 2015 Reliability Assessment

June 2016 First Steps into the Bootstrap World

October 2015 2014 Reliability Assessment

August 2014 Empirical Signal-to-Noise Ratios for Operational Test Data

August 2014 2013 Reliability Assessment

Selected Publications

Peer Reviewed Journals

Freeman L. J., Johnson, T., Avery, M., Lillard, V., Clutter, J. “Testing Defense Systems.” Analytic Methods in Systems and Software Testing. © 2018, pages 439-487 John Wiley & Sons, Ltd. Editors: Kenett, R., Ruggeri, F., Faltin, F.

Avery, M., Avery, K., Freeman, L.J. “Statistical Methods for Defense Testing.” Wiley Statistics Reference Online (2018).

Avery, M., Orndorff, M., Robinson, T., & Freeman, L.J. “Regularization for Continuously Observed Ordinal Response Variables with Piecewise‐constant Functional Covariates.” Quality and Reliability Engineering International (2016) 32(6).

J. L. Bell, M. R. Avery and M. C. Wells, “Reliability survey of military acquisition systems,” 2015 Annual Reliability and Maintainability Symposium (RAMS), Palm Harbor, FL, 2015, pp. 1-6.

Haslauer, C. M., Avery, M. R., Pourdeyhimi, B., & Loboa, E. G. (2015). “Translating Textiles to Tissue Engineering: Creation and Evaluation of Microporous, Biocompatible, Degradable Scaffolds Using Industry Relevant Manufacturing Approaches and Human Adipose Derived Stem Cells.” Journal of Biomedical Materials Research. Part B, Applied Biomaterials, 103(5), 1050–1058.

Avery M., Wu Y., Zhang H., Zhang J. “RKHS-based functional nonparametric regression for sparse and irregular longitudinal data.” Canadian Journal of Statistics, (2014) 42 (2) , pp. 204-216.

Avery, M. & Banks, H. & Basu, K. & Cheng, Y. & Eager, E. & Khasawinah, S. & Potter, L. & Rehm, K. (2012). “Experimental design and inverse problems in plant biological modeling.” Journal of Inverse and Ill-Posed Problems. 20. 169–191.

Peer Reviewed Technical Reports

Pechacek, Julie A., Avery, Matthew R., Bishop, James M., Picucci, P. M., Romana, Amrit K. and Saizan, Alexandra M. “Design for a Controlled Trial of an Alcohol Market Intervention Intended to Reduce Harmful Behaviors in the Army and Air Force”. (2018) IDA Paper P-9185

Brooks, Peter S., Avery, Matthew R., Avery, Kelly M., Moses, Franklin L. and Thorne, James D. “(U) Effectiveness of ISR Assets Against Hard Targets”. (2018) IDA Document NS D-9145, -9146, and -9147

Thomas, Dean, Goodman, Allison L., Avery, Matthew R., Bell, Jonathan L., Dickinson, Rebecca M., Hueckstaedt, Robert M., Peek, Douglas A., Roberts, Max W. and Freeman, Laura J. “Assessment of Reliability in DoD Acquisition Programs”. (2018) IDA Document NS D-8997

Schulman, Eric R., Avery, Matthew R., Corbett, Brooke M., Porter, Daniel J., Robbins, William J., Wall, Kenneth E., Wojton, Heather M. and Luhman, Michael L. “(U) Independent Evaluation of the Patriot Post-Deployment Build-8 Initial Operational Test and Evaluation”. (2018) IDA Document D-8665

Freeman, Laura J. and Avery, Matthew R. “Prediction Uncertainty for Autocorrelated Lognormal Data with Random Effects”. (2017) IDA Document NS D-8629

Peterson, Robert G., Schulman, Eric R., Avery, Matthew R., Schlick, Conor P. and Filkins, Peter C. “(U) Independent Evaluation of the Joint Standoff Weapon C-1 Follow-on Operational Test and Evaluation”. (2017). IDA Paper P-8369

Shaw, Scott E., Avery, Matthew R., Crabtree, Brent A., Grandinetti, Saul A. and Lambrecht, Kristen L. “RQ-7BV2 Shadow Tactical Unmanned Aircraft System Follow-on Operational Test and Evaluation, and Operational Assessment of the One System Remote Video Terminal”. (2015). IDA Document D-5373

Shaw, Scott E., Avery, Matthew R., Brathwaite, Joy D., Eveker, Kevin M. and Lambrecht, Kristen L. “(U) Independent Assessment of the RQ-21A Blackjack Small Tactical Unmanned Aircraft System - Volume II - Appendix C, (U) RQ-21A Blackjack Cybersecurity Assessment”. (2016). IDA Paper P-5270 (Volumes 1 and 2)

Carlson, Charles D., Satyapal, Shanti, Ratchford, Jasmine, Avery, Matthew R. and Kirpichevsky, Yevgeniy. “Test and Evaluation Concept for the Biometrics Enabling Capability (BEC) Increment 0 IOT&E”. (2014). IDA Document D-5078

Shaw, Scott E. and Avery, Matthew R. “Land-Based Phase of the MQ-8B Weapons Quick Reaction Assessment”. (2013). IDA Document D-5051

Thayer, Bradley D., Mathiasmeier, Kenneth J. and Avery, Matthew R. “Test and Evaluation Concept for the Next Generation Jammer (NGJ)”. (2014). IDA Document D-4972

Avery, Matthew R. and Shaw, Scott E. “Assessment of RQ-21A Small Tactical Unmanned Aircraft System Performance During Operational Test Period B2”. (2013). IDA Document D-4904

Memberships

International Test and Evaluation Association (ITEA)

American Statistical Association