Research
Extramural Grants
National Science Foundation, DRL-2000705, “A Statistical Framework and Tools for Planning Multilevel Randomized Cost-Effectiveness Trials”, Principal Investigator (PI/Co-PIs: Nianbo Dong, Rebecca Maynard, Benjamin Kelcey, Jessaca Spybrook, Wei Li), 9/1/2020 – 8/31/2024, $1,274,904.
U.S. Department of Education, Institute of Education Sciences, Statistical and Research Methodology in Education, R305D190013, “Empirical Benchmarks for Interpreting Effect Size and Design Parameters for Planning Multilevel Randomized Trials on Social & Behavioral Outcomes”, Principal Investigator (PI/Co-PIs: Nianbo Dong, Sandra Wilson, Keith Herman, Wendy Reinke), 8/1/2019 – 7/31/2024, $893,955.
National Science Foundation, DGE-1760884, “Multisite Designs for Teacher Development Processes in Mathematics”, Co-Principal Investigator (PI/Co-PIs: Benjamin Kelcey, Jessaca Spybrook, Nianbo Dong), 9/1/2018 – 8/31/2024, $499,996.
National Science Foundation, DRL-1913563 (1651153), “CAREER: Design of Multisite Moderation Studies to Examine the Factors that Explain Treatment Effect Variation on Student Mathematics Achievement in Teacher Professional Development”, Principal Investigator, 1/15/2017 – 12/31/2024, $553,218.
U.S. Department of Education, Institute of Education Sciences, Statistical and Research Methodology in Education, R305D150001, “Estimating Population Effects: Incorporating Propensity Scores with Complex Survey Data”, Co-Principal Investigator (PI/Co-PI: Elizabeth Stuart, Nianbo Dong), Campus PI (Subcontract from Johns Hopkins University, $298,943), 7/1/2015 – 6/30/2018, 7/1/2018 – 6/30/2019 (no cost extension), $798,002.
U.S. Department of Education, Institute of Education Sciences, R305B150028, “Missouri Interdisciplinary Postdoctoral Research and Training Program”, Co-Principal Investigator (PI/Co-PIs: Wendy Reinke, Keith Herman, Timothy Riley-Tillman, Nianbo Dong), 7/1/2015 – 6/30/2020, $699,561.
U.S. Department of Education, Institute of Education Sciences, R305A150517, “Evaluation of a Self-Monitoring Training Program for Elementary School Students”, Co-Investigator (PI/Co-PIs: Aaron Thompson, Wendy Reinke, Keith Herman), 8/1/2015-7/31/2019, $3,499,927.
National Science Foundation, DGE-1437745, “Collaborative Research: Power Analyses for Moderator and Mediator Effects in Cluster Randomized Trials”, Principal Investigator, 8/15/2014 – 8/31/2017, 9/1/2017-8/31/2019 (no cost extension), $243,552. Effort: 100%. (Other PIs in collaborative universities: Benjamin Kelcey at University of Cincinnati and Jessaca Spybrook at Western Michigan University, total award for this collaborative project: $799,688).
National Science Foundation, DRL-1414438, “From elementary generalist to mathematics specialist: Examining teacher practice and student outcomes in departmental and self-contained models”, Co-Principal Investigator (PI/Co-PIs: Barbara Reys, James Tarr, Corey Webel, Nianbo Dong), 8/1/2014 – 7/31/2018, $2,288,762.
U.S. Department of Education, Institute of Education Sciences, R305A130143, “Evaluation of a Classroom Management Training Program for Middle School Teachers”, Co-Investigator (PI/Co-PI: Keith Herman, Wendy Reinke), 08/01/2013 – 07/31/2017. $2,998,403.
Internal Grants
UNC System Office, “Evaluation of the Effectiveness and Impact of Learning Assistants”, Principal Investigator, 04/01/2019 – 12/31/2021. $27,556.
UNC Office of Research Development (ORD) IDEA Grant, “Adaptive Intelligent Models to Promote Science Learning for Linguistically Diverse Students “, Co-Principal Investigator (PI: Kihyun “Kelly” Ryoo). (July 1, 2019 – June 30, 2020). $19,995.
UNC-CH – SOE, “Building a Contextual Community of Practice around Education Data Sciences: Heightening Capacity for Mixed Methods Research”, Principal Investigator (Co-PIs: Alison LaGarry, Diana Lys). (May 11, 2019 – December 31, 2019). $10,000.
UNC-CH – SOE, “Improving Science Learning for Linguistically Diverse Students Using Automated Feedback and Data Visualizations “, Co-Principal Investigator (PI: Kihyun “Kelly” Ryoo). (June 1, 2019 – December 31, 2019). $10,000.
UNC-CH – SOE, “Evidence Based Staffing: A Feasibility Study and Proposal Completion Project”, Co-Principal Investigator (PI: Matthew G. Springer). (June 1, 2019 – December 31, 2020). $19,999.
University of Missouri Research Council Grants, “An Investigation of the Mechanism through Which the Class Size Reduction Policy Succeed in Some Countries: A Multilevel Mediation Analysis”, Principal Investigator (PI/Co-PI: Nianbo Dong, Wei Li), 02/01/2016 – 05/31/2017. $1,000.
Other Research Experience
Research Associate, Peabody Research Institute, Vanderbilt University, 2009-2013
(1) U.S. Department of Education’s Institute of Education Sciences grant, “Evaluating the Effectiveness of Tennessee’s Voluntary Pre-K Program”, Mark Lipsey, Principal Investigator.
(2) U.S. Department of Education’s Institute of Education Sciences grant, “Learning-Related Cognitive Self-Regulation School Readiness Measures for Preschool Children: Optimizing Predictive Validity for Achievement”, Mark Lipsey, Principal Investigator.
(3) U.S. Department of Education’s Institute of Education Sciences grant, “Scaling Up TRIAD: Teaching Early Mathematics for Understanding with Trajectories and Technologies”, Mark Lipsey, Principal Investigator (TN site).
Research Assistant, University of Pennsylvania, 2006-2009
- Annenberg Foundation grant, “Evaluation of Annenberg Distributed Leadership Teacher Training Program”, Rebecca Maynard, Principal Investigator.
Peer Reviewed Publications
Dong, N., Kelcey, B., & Spybrook, J. (2023). Experimental design and power for moderation in multisite cluster randomized trials. Advance online publication. The Journal of Experimental Education. Advance online publication. doi: 10.1080/00220973.2023.2226934
Li, W., Dong, N., Maynard, R. A., Spybrook, J., & Kelcey, B. (2022). Experimental Design and Statistical Power for Cluster Randomized Cost-Effectiveness Trials. Journal of Research on Educational Effectiveness. Advance online publication. doi: 10.1080/19345747.2022.2142177
Dong, N., Herman, K. C., Reinke, W. M., Wilson, S. J., & Bradshaw, C. P. (2022). Gender, racial, and socioeconomic disparities on social and behavioral skills for K-8 students with and without interventions: An integrative data analysis of eight cluster randomized trials. Prevention Science. Advance online publication. doi: 10.1007/s11121-022-01425-w (https://rdcu.be/cT9qH)
Bulus, M., & Dong, N. (2022). Consequences of ignoring a level of nesting on design and analysis of blocked three-level regression discontinuity designs: Power and Type I error rates. Adiyaman University Journal of Educational Sciences, 12(1), 42-55. doi: https://doi.org/10.17984/adyuebd.1068923
Dong, N., Curenton, S. M., Bulus, M., & Ibekwe-Okafor, N. (2022). Investigating the differential effects of early child care and education in reducing gender and racial academic achievement gaps from kindergarten to 8th grade. Journal of Education. Advance online publication. doi: 10.1177/00220574221104979
Dong, N., Kelcey, B., & Spybrook, J. (2022). Identifying and estimating causal moderation for treated and targeted subgroups. Multivariate Behavioral Research. Advance online publication. doi: 10.1080/00273171.2022.2046997
Herman, K. C., Dong, N., Reinke, W. M., & Bradshaw, C. P. (2022). Accounting for traumatic historical events in randomized controlled trials. School Psychology Review. Advance online publication. doi: https://doi.org/10.1080/2372966X.2021.2024768
Herman, K. C., Reinke, W. M., Dong, N., & Bradshaw, C. (2022). Can effective classroom behavior management increase student achievement in middle school? Findings from a group randomized trial. Journal of Educational Psychology, 114(1), 144–160. https://doi.org/10.1037/edu0000641
Dong, N., Kelcey, B., & Spybrook, J. (2021). Design considerations in multisite randomized trials to probe moderated treatment effects. Journal of Educational and Behavioral Statistics, 46 (5), 527-559. doi: 10.3102/1076998620961492 (pdf; SAS code for estimating design parameters; Example dataset; PowerUp!-MRTs)
Dong, N., Spybrook, J., Kelcey, B., & *Bulus, M. (2021). Power analyses for moderator effects with (non)random slopes in cluster randomized trials. Methodology, 17 (2), 92-110. doi: https://doi.org/10.5964/meth.4003
*Bulus, M., & Dong, N. (2021). Bound constrained optimization of sample sizes subject to monetary restrictions in planning multilevel randomized trials and regression discontinuity studies. The Journal of Experimental Education, 89 (2), 379-401. doi: 10.1080/00220973.2019.1636197
Kelcey, B., Cox, K., & Dong, N. (2021). Croon’s bias-corrected factor score path analysis for small to moderate sample multilevel structural equation models. Organizational Research Methods, 24 (1), 55-77. doi: 10.1177/1094428119879758
Reinke, W. M., Stormont, M., Herman, K. C., & Dong, N. (2021). The Incredible Years Teacher Classroom Management Program: Effects for students receiving special education services. Remedial and Special Education, 42 (1), 7-17. doi: 10.1177/0741932520937442
Sinclair, J., Reinke, W. M., Herman, K. C., Dong, N., & Stormont, M. (2021). Effects of a universal classroom management intervention on middle school students at risk for behavior problems. Remedial and Special Education, 42 (1), 18-30. doi: 10.1177/0741932520926610
Dong, N., Stuart, E. A., Lenis, D., & Nguyen, T. Q. (2020). Using propensity score analysis of survey data to estimate population average treatment effects: A case study comparing different methods. Evaluation Review, 44 (1) 84-108. doi: 10.1177/0193841X20938497
Spybrook, J., Zhang, Q., Kelcey, B., & Dong, N. (2020). Learning from cluster randomized trials in education: An assessment of the capacity of studies to determine What Works, For Whom, and Under What Conditions. Educational Evaluation and Policy Analysis, 42 (3), 354–374. doi: 10.3102/0162373720929018
*Li, W., Dong, N., & Maynard, R. A. (2020). Power analysis for two-level multisite randomized cost-effectiveness trials. Journal of Educational and Behavioral Statistics, 45 (6), 690-718. doi: 10.3102/1076998620911916
Kelcey, B., Spybrook, J., Dong, N., & Bai, F. (2020). Cross-level mediation in school-randomized studies of teacher development: Experimental design and power. Journal of Research on Educational Effectiveness, 13 (3), 459-487. doi: 10.1080/19345747.2020.1726540
Kelcey, B., Xie, Y., Spybrook, J., & Dong, N. (2020). Power and sample size determination for multilevel mediation in three-level cluster-randomized trials. Multivariate Behavioral Research. Advance online publication. doi: 10.1080/00273171.2020.1738910
Dong, N. & Kelcey, B. (2020). A review of Causality in a Social World: Moderation, Mediation, and Spill-over. Journal of Educational and Behavioral Statistics, 45 (3), 374–378. doi: 10.3102/1076998619881791×10060
Wiedermann, W., Dong, N., & von Eye, A. (2019). Advances in statistical methods for causal inference in prevention science: Introduction to the special section. Prevention Science, 20 (3), 390-393. doi: 10.1007/s11121-019-0978-x
Kelcey, B., Spybrook, J., & Dong, N. (2019). Sample size planning in cluster-randomized interventions probing multilevel mediation. Prevention Science, 20 (3), 707-418. doi: 10.1007/s11121-018-0921-6
Lenis, D., Nguyen, T. Q., Dong, N., & Stuart, E. A. (2019). It’s all about balance: Propensity score matching in the context of complex survey data. Biostatistics, 20(1):147-163. doi:10.1093/biostatistics/kxx063
Reinke, W. M., Herman, K. C., & Dong, N. (2018). The Incredible Years Teacher Classroom Management program: Outcomes from a group randomized trial. Prevention Science, 19(8), 1043–1054. doi: 10.1007/s11121-018-0932-3
Dong, N., & Lipsey, M. W. (2018). Can propensity score analysis approximate experiments using pretest and demographic information in Pre-K intervention research? Evaluation Review, 42(1), 34-70. doi: 10.1177/0193841X17749824
Dong, N., Kelcey, B., & Spybrook, J. (2018). Power analyses of moderator effects in three-level cluster randomized trials. Journal of Experimental Education, 86 (3), 489-514. doi: 10.1080/00220973.2017.1315714
Kelcey, B., Dong, N., Spybrook, J., & Shen, Z. (2017). Experimental Power for Indirect Effects in Group-randomized Studies with Group-level Mediators. Multivariate Behavioral Research, 52 (6), 699-719. doi: 10.1080/00273171.2017.1356212
Kelcey, B., Dong, N., Spybrook, J., & Cox, K. (2017). Statistical power for causally-defined individual and contextual indirect effects in group-randomized Trials. Journal of Educational and Behavioral Statistics, 24 (5), 499-530. doi: 10.3102/1076998617695506
Lipsey, M. W., Nesbitt, K. T., Farran, D. C., Dong, N., Fuhs, M. W., & Wilson, S. J. (2017). Learning-related cognitive self-regulation measures for prekindergarten children: A comparative evaluation of the educational relevance of selected measures. Journal of Educational Psychology, 109(8), 1084-1102. doi: 10.1037/edu0000203
Dong, N., Reinke, W. M., Herman, K. C., Bradshaw, C. P., & Murray, D. W. (2016). Meaningful effect sizes, intraclass correlations, and proportions of variance explained by covariates for panning two- and three-level cluster randomized trials of social and behavioral outcomes. Evaluation Review, 40(4), 334-377. doi: 10.1177/0193841X16671283
Spybrook, J., Kelcey, B., & Dong, N. (2016). Power for detecting treatment by moderator effects in two and three-level cluster randomized trials. Journal of Educational and Behavioral Statistics, 41(6), 605-627. doi: 10.3102/1076998616655442
Curenton, S. M., Dong, N., & Shen, X. (2015). Does aggregate school achievement mediate long-term outcomes for early childhood education participants? Developmental Psychology, 51(7), 921-34. doi: 10.1037/a0039295.
Dong, N. (2015). Using propensity score methods to approximate factorial experimental designs to analyze the relationship between two variables and an outcome. American Journal of Evaluation, 36(1), 42-66. doi: 10.1177/1098214014553261.
Huang, C., & Dong, N. (2014). Dimensionality of the Children’s Depression Inventory: Meta-analysis of pattern matrices. Journal of Child and Family Studies, 23(7), 1182-1192. doi: 10.1007/s10826-013-9779-1.
Fuhs, M. W., Turner, K. A., Farran, D. C., & Dong, N. (2014). Longitudinal associations between executive functioning and academic skills across content areas. Developmental Psychology, 50(6), 1698-1709. doi: 10.1037/a0036633.
Dong, N., & Maynard, R. A. (2013). PowerUp!: A tool for calculating minimum detectable effect sizes and minimum required sample sizes for experimental and quasi-experimental design studies. Journal of Research on Educational Effectiveness, 6(1), 24-67.
doi: 10.1080/19345747.2012.673143.
PowerUp! is available at: http://www.causalevaluation.org/
Note: Winner of the 2013 AERA Division H (Research, Evaluation, and Assessment in Schools) Outstanding Publication Award, Advances in Methodology, for this article.
Hawkinson, L., Griffen, A. S., Dong, N., & Maynard, R. A. (2013). The relationship between child care subsidies and children’s cognitive development. Early Childhood Research Quarterly, 28 (2), 388-404. http://dx.doi.org/10.1016/j.ecresq.2012.10.002
Dong, N., & Cravens, X. (2012). Leadership, learning-centered school conditions, and mathematics achievement: What can the U.S. learn from top performers from TIMSS? IERI Monograph Series: Issues and Methodologies in Large-Scale Assessments, 5, 79-113.
Huang, C., & Dong, N. (2012). Factor structures of the Rosenberg self-esteem scale: Meta-analysis of pattern matrices. European Journal of Psychological Assessment, 28(2), 132-138. doi: 10.1027/1015-5759/a000101.
Dong, N., Maynard, R. A. and Perez-Johnson, I. (2008). Averaging effect sizes within and across studies of interventions aimed at improving child outcomes. Child Development Perspectives, 2(3), 187-197. doi: 10.1111/j.1750-8606.2008.00064.x.
BOOK CHAPTER
Zhang, Q., Spybrook, J., Kelcey, B., & Dong, N. (2023). Foundational methods: power analysis. In Tierney, R.J., Rizvi, F., Erkican, K. (Eds.), International Encyclopedia of Education, vol. 14 (pp. 784-791). Elsevier. https://dx.doi.org/10.1016/B978-0-12-818630-5.10088-0.
Spybrook, J., Kelcey, B., & Dong, N. (2022). Statistical power for linear multilevel models. In A. O’Connell, B. McCoach, & B. Bell. (Eds.), Multilevel Modeling Methods with Introductory and Advanced Applications (pp. 127-164). Charlotte, NC: Information Age Publishing.
Invited Presentations & Workshops
Dong, N. (2023, June). Multilevel Randomized Cost-Effectiveness Trials. Presentation at the NIH funded 2023 Short Course “Strengthening Causal Inference in Behavioral Obesity Research”, Indiana University School of Public Health-Bloomington.
Dong, N. (2022, September). Gender, racial, and socioeconomic disparities on social and behavioral skills for K-8 students with and without interventions: An integrative data analysis of eight cluster randomized trials. Presentation at the Prevention Science Seminar, Missouri Prevention Science Institute, University of Missouri.
Dong, N. (2022, June). Power Analysis: A Critical Step in Program Evaluation. Presentation at the international (online) summer school on “Frontier in Economics of Education and Chinese Practice”, Graduate School of Education, Peking University.
Dong, N. (2021, June). Designing Adequately Powered Cluster and Multisite Randomized Trials to Detect Main Effects, Moderation, and Mediation. Workshop sponsored by the University of Pennsylvania IES Predoctoral Training Program.
Dong, N. (2016, January). Welcome to PowerUp!: A tool for calculating Minimum Detectable Effect Sizes and Minimum Required Sample Sizes for experimental and quasi-experimental design studies. Webinar presented at Technical Evaluation Assistance in Mathematics and Science (TEAMS), RMC Research Corporation.
Dong, N. (2015, April). How well propensity score methods approximate experiments using pretest and demographic information in educational research? The Quantitative Psychology Colloquium Series, Department of Psychological Sciences, University of Missouri.
Dong, N. (2014, April). The short- and long-term effects of early child care in reducing black-white and Hispanic-white math achievement gaps: An application of generalized propensity score methods in approximating factorial experiments. Population, Education, and Health Seminar Series, Co-sponsored by the Institute of Public Policy and the Truman School of Public Affairs, University of Missouri.
Dong, N. & Maynard, R. (2013, May). Applications of power analysis in the design of experimental and quasi-experimental evaluations. Seminar and Workshop Presentations at American Institutes for Research, Washington DC.
Dong, N. & Maynard, R. (2013, May). Applications of power analysis in the design of experimental and quasi-experimental evaluations. Seminar and Workshop sponsored by the University of Pennsylvania IES Predoctoral Training Program.
Conference Presentations & Workshops (*indicates student/postdoc advisee)
Dong, N., Herman, K. C., Reinke, W. & Ren, S. (2023, September). Heterogeneity of Intraclass Correlation Coefficients (ICCs) among Social & Behavioral Outcomes and School Poverty Levels. Paper accepted to present at the 2023 Annual Conference of the Society for Research on Educational Effectiveness (SREE), Arlington, VA.
Dong, N., Spybrook, J. & Kelcey, B. (2023, September). Statistical Power for Moderation in Three-level Multisite Individual Randomized Trials and Consequences of Ignoring a Level of Nesting. Paper accepted to present at the 2023 Annual Conference of the Society for Research on Educational Effectiveness (SREE), Arlington, VA.
Ataneka, A., Xie, Y., Kelcey, B., & Dong, N. (2023, September). Design Parameter Values for Multilevel Mediation Studies of Teacher Development. Paper accepted to present at the 2023 Annual Conference of the Society for Research on Educational Effectiveness (SREE), Arlington, VA.
Li, W., Dong, N., Maynard, R. A., Kelcey, B., & Spybrook, J. (2023, September). Power Analysis for Multisite Randomized Cost Effectiveness Trials with Site Fixed Effects. Paper accepted to present at the 2023 Annual Conference of the Society for Research on Educational Effectiveness (SREE), Arlington, VA.
Li, W., Dong, N., Maynard, R. A., Kelcey, B., Spybrook, J., & Xu, Y. (2023, June). Power analysis and sample size planning in the design of two-level randomized cost-effectiveness trials. Paper presented at the 2023 Modern Modeling Methods Conference, Mansfield, CT.
Kelcey, B., Dong, N., & Spybrook, J. (2023, April). Designing Adequately Powered Cluster and Multisite Randomized Trials to Detect Main Effects, Moderation, and Mediation. Workshop conducted at the 2023 Annual Meeting of the American Educational Research Association, Chicago, IL.
Dong, N., Kelcey, B. & Spybrook, J. (2023, April). Experimental Design and Power for Moderation in Multisite Cluster Randomized Trials to Investigate Treatment Effect Heterogeneity. Paper presented at the 2023 Annual Meeting of the American Educational Research Association, Chicago, IL.
Westine, C. D., Wu, T., Dong, N., Maher, D. & Kim, S. (2023, April). Evaluation of the Effectiveness of a University Learning Assistant Program. Paper presented at the 2023 Annual Meeting of the American Educational Research Association, Chicago, IL
Dong, N., Li, W., Maynard, R., Spybrook, J., Kelcey, B. & Bowden, A. B. (2022, November). Advantages of Monte Carlo Confidence Intervals for Incremental Cost-Effectiveness Ratios: A Comparison of Five Methods. Paper presented at the 44th Annual Association for Public Policy Analysis and Management (APPAM), Washington, DC.
Dong, N., Li, W., Maynard, R., Spybrook, J., Kelcey, B. & Bowden, A. B. (2022, September). Advantages of Monte Carlo Confidence Intervals for Incremental Cost-Effectiveness Ratios: A Comparison of Five Methods. Paper presented at the 2022 Annual Conference of the Society for Research on Educational Effectiveness (SREE), Arlington, VA.
Dong, N., Herman, K., Reinke, W. & Wilson, S. (2022, April). Design Parameters for Planning Multilevel Randomized Trials on Social & Behavioral Outcomes. Paper presented at the 2022 Annual Meeting of the American Educational Research Association, San Diego, CA.
Dong, N., Herman, K., Reinke, W. & Wilson, S. (2022, March). Disparities on Social and Behavioral Outcomes Between Gender, Race, and Social Economics Status for K-8 Students With and Without Interventions: Findings from Eight Cluster Randomized Trials. Poster accepted to present at the 43rd Annual Association for Public Policy Analysis and Management (APPAM).
Dong, N., Herman, K., Reinke, W., & Wilson, S. (2021). Disparities on Social and Behavioral Outcomes Between Gender, Race, and Social Economics Status for K-8 Students With and Without Interventions: Findings from Eight Cluster Randomized Trials. Paper accepted to present at the 43rd Annual Association for Public Policy Analysis and Management (APPAM).
Dong, N., Kelcey, B., & Spybrook, J. (2021, September). Estimating the Average Moderated Treatment Effects on the Treated and Targeted Subgroups. Paper presented at the 2021 Virtual Conference of the Society for Research on Educational Effectiveness (SREE).
Dong, N., Herman, K., Reinke, W., & Wilson, S. (2021, September). Design Parameters for Planning Multilevel Randomized Trials and Empirical Benchmarks for Interpreting Effect Size on Social & Behavioral Outcomes. Paper presented at the 2021 Virtual Conference of the Society for Research on Educational Effectiveness (SREE).
Li, W., Dong, N., Kelcey, B., Spybrook, J. & Maynard, R. (2021, September). Optimal Design in Two-Level Cluster Randomized Cost-Effectiveness Trials. Paper accepted to present at the 2021 Virtual Conference of the Society for Research on Educational Effectiveness (SREE).
Dong, N., Kelcey, B., & Spybrook, J. (2021, July). Designing Adequately Powered Cluster and Multisite Randomized Trials to Detect Main Effects, Moderation, and Mediation. Workshop conducted at the AERA 2021 Virtual Research Learning Series.
Kelcey, B., Bai, F., Spybrook, J., & Dong, N. (2021, April). Experimental Power for Cross-Level Mediation in School-Randomized Studies of Teacher Development. Paper presented at the 2021 Virtual AERA Annual Meeting.
Dong, N. (2020, November). Design Considerations in Multisite Randomized Trials (MRTs) Probing Moderated Treatment Effects. Virtual poster presented at the 2020 NSF HER Core Research (ECR) meeting.
Dong, N., Kelcey, B., & Spybrook, J. (2020, March). Estimation of Causal Moderated Treatment Effects Under the Potential Outcomes Framework: An Application of the Propensity Score. Paper to be presented at the Spring 2020 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Dong, N., Kelcey, B., & Spybrook, J. (2019, November). Causal Moderation Analysis Based on Potential Outcomes Framework. Paper presented at Evaluation 2019, the annual conference of the American Evaluation Association, Minneapolis, MN.
*Li, W., Dong, N., & Maynard, M. (2019, November). Power Analysis for Two- and Three- Level Cluster Randomized Cost Effectiveness Trials with Binary Effectiveness Measures. Paper presented at Evaluation 2019, the annual conference of the American Evaluation Association, Minneapolis, MN.
Spybrook, J., Zhang, Q., Kelcey, B., & Dong, N. (2019, November). What works? For Whom? And Under What Conditions?An Analysis of Key Individual and School Level Moderators and the Precision of Studies to Assess Moderator Effects. Paper presented at Evaluation 2019, the annual conference of the American Evaluation Association, Minneapolis, MN.
*Li, W., Dong, N., & Maynard, M. (2019, November). Power Analysis for Cluster Randomized Cost Effectiveness Trials with Binary Effectiveness Measures. Paper presented at the 41st Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Denver, CO.
Dong, N., Kelcey, B., & Spybrook, J. (2019, April). Power Analysis of Moderation Effects in Multisite Randomized Trials. Paper presented at the 2019 Annual Meeting of the American Educational Research Association, Toronto, ON, Canada.
Kelcey, B., Dong, N., Spybrook, J., & Cox, K. T. (2019, April). Designing Adequately Powered Cluster Randomized Trials to Detect Main Effects, Moderation, and Mediation. Workshop conducted at the 2019 Annual Meeting of the American Educational Research Association, Toronto, ON, Canada.
Kelcey, B., Cox, K. T., & Dong, N. (2019, April). Unbiased Estimation of Multilevel Structural Equation Models in Small to Moderate Sample Sizes: A Limited Information Estimator. Paper presented at the 2019 Annual Meeting of the American Educational Research Association, Toronto, ON, Canada.
Dong, N., Kelcey, B., & Spybrook, J. (2019, March). Power Analysis of Two- and Three-Level Multisite Moderation Studies. In N. Dong (Chair), Design Considerations and Challenges in Planning Complicated Multilevel Experiments. Symposium presented at the Spring 2019 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Kelcey, B., Cox, K. T., & Dong, N. (2019, March). A Bias-Corrected Limited Information Estimator for Small to Moderate Scale Multilevel Structural Equation Models. Paper presented at the Spring 2019 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Dong, N., Spybrook, J., & Kelcey, B. (2018, November). Design of Three-Level Multisite Moderation Studies with Adequate Power to Investigate Treatment Effect Variation. Poster presented at the 40th Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Washington, DC.
Kelcey, B., Cox, K., & Dong, N.. (2018, November). Apportioned Multilevel Structural Equation Modeling: An Unbiased and Efficient Alternative to Multilevel Structural Equation Modeling in Small to Moderate Samples. Paper presented at the 40th Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Washington, DC.
*Bulus, M., Dong, N., Kelcey, B., & Spybrook, J. (2018, April). A Web-based Application for Constrained Optimal Sample Allocation in Design of Multilevel Randomized Experiments. Roundtable presented at the 2018 Annual Meeting of the American Educational Research Association, New York, NY.
Spybrook, J., Kelcey, B., Dong, N., & Cox, K. T. (2018, April). Designing Adequately Powered Cluster Randomized Trials to Detect Main Effects, Moderation, and Mediation. Workshop presented at the 2018 Annual Meeting of the American Educational Research Association, New York, NY.
Kelcey, B., Cox, K. T., & Dong, N. (2018, April). Correcting for Unreliability in Power Analyses of Cluster-Randomized Trials of Mediation. Paper presented at the 2018 Annual Meeting of the American Educational Research Association, New York, NY.
*Li, W., Dong, N., & Maynard, M. (2018, March). Power Analysis for Three Level Blocked Randomized Cost Effectiveness Trials. Paper presented at the Spring 2018 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Dong, N., Stuart, E. A., Lenis, D., & Nguyen, T. (2018, January). Using Propensity Score Analysis of Survey Data to Estimate Population Average Treatment Effects. Poster presented at the NCER/NCSER Annual Principal Investigators (PI) Meeting. Arlington, VA.
Maynard, R. A., Dong, N., Kelcey, B., & Spybrook, J. (2017, November). Power Analysis of Multi-level and Single-Level Experiments Using PowerUp!. Workshop presented at Evaluation 2017, the annual conference of the American Evaluation Association, Washington, DC.
Spybrook, J., Kelcey, B., & Dong, N. (2017, November). Designing Adequately Powered Studies of Multilevel Moderation and Mediation using PowerUp!. Workshop presented at Evaluation 2017, the annual conference of the American Evaluation Association, Washington, DC.
*Li, W., Dong, N., & Maynard, M. (2017, November). Power Analysis for Multilevel Cluster Randomized Cost Effectiveness Trials. Paper presented at the 39th Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Chicago, IL.
Kelcey, B., Dong, N., Spybrook, J., & Shen, Z. (2017, November). Power Analyses for Cluster, Contextual, and Individual Mediation in Hierarchical Experiments. Paper presented at the 39th Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Chicago, IL.
Dong, N. (2017, September). Design Multisite Moderation Studies to Investigate Treatment Effect Variation in STEM Studies. Poster presented at the National Science Foundation’s Education and Human Resources Core Research (ECR) Program PI Convening for Fundamental Research in STEM Education: Progress, Issues, & the Future. Alexandria, VA.
Kelcey, B., Dong, N., Spybrook, J., & Cox, K. T. (2017, April). Statistical Power for Individual and Contextual Mediation in Cluster-Randomized Designs. Paper presented at the 2017 Annual Meeting of the American Educational Research Association, San Antonio, TX.
Dong, N., Spybrook, J., & Kelcey, B. (2017, March). PowerUp!-Moderator: A software assisting the design of cluster randomized trials to detect the moderator effects. In J. Rickles (Chair), Expanding the Power Analysis Toolkit: A Description and Demonstration of New Resources. Symposium conducted at the Spring 2017 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
*Bulus, M., & Dong, N. (2017, March). Prognostic propensity scores: A method accounting for the correlations of the covariates with both the treatment and the outcome variables in matching and diagnostics. Poster presented at the Spring 2017 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
*Bulus, M., & Dong, N. (2017, March). Constrained optimal sample allocation in multilevel experiments using PowerUpR. Paper presented at the Spring 2017 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
*Li, W., Dong, N., & Maynard, M. (2017, March). Power analysis for multilevel randomized cost effectiveness trials. Paper presented at the Spring 2017 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Kelcey, B., Dong, N., & Spybrook, J. (2017, March). PowerUp!-Mediator: Software for designing group-randomized studies of mediation. In J. Rickles (Chair), Expanding the Power Analysis Toolkit: A Description and Demonstration of New Resources. Symposium conducted at the Spring 2017 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Dong, N., Kelcey, B., Spybrook, J. & Cox, K. (2017, January). Design of multilevel studies of mediation and moderation. Workshop presented at the 2017 Hawaii International Conference on Education, Honolulu, HI.
Stuart, E. & Dong, N. (2016, April). Combining propensity score methods and complex survey data to estimate population treatment effects. In D. Swan (Chair), Leveraging federal data in education research: Engaging with administrative and complex sample data. Symposium conducted at the 2016 Annual Meeting of the American Educational Research Association, Washington, DC.
Dong, N., Spybrook, J., & Kelcey, B. (2016, March). A general framework for power analysis to detect the moderator effects in two- and three-level cluster randomized trials. In N. Dong (Chair), Advances in power analysis to detect the moderator, mediator, and polynomial effects in multilevel experiments. Symposium conducted at the Spring 2016 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Kelcey, B., Dong, N., Spybrook, J. & Shen, Z. (2016, March). Power analyses for causally-defined conditional indirect effects in multilevel designs. In N. Dong (Chair), Advances in power analysis to detect the moderator, mediator, and polynomial effects in multilevel experiments. Symposium conducted at the Spring 2016 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Spybrook, J., Kelcey, B., & Dong, N. (2016, March). Power calculations for moderators in multi-site cluster randomized trials. In N. Dong (Chair), Advances in power analysis to detect the moderator, mediator, and polynomial effects in multilevel experiments. Symposium conducted at the Spring 2016 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Stuart, E., Dong, N., & Lenis, D. (2016, March). Combining propensity score methods and complex survey data to estimate population treatment effects. Paper presented at the Spring 2016 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Kelcey, B., Spybrook, J., & Dong, N. (2016, January). Designing multilevel studies of mediation and moderation. Workshop presented at the 2016 Hawaii International Conference on Education, Honolulu, HI.
Dong, N., Spybrook, J., & Kelcey, B. (2015, November). Power analysis to detect the continuous moderator effects in cluster randomized trials. Paper presented at the 37th Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Miami, FL.
Dong, N. & *Bulus, M. (2015, April). The short- and long-term effects of early childhood education in reducing child math achievement gaps: An application of generalized propensity score methods in approximating factorial experiments. Poster session presented at the 2015 Annual Meeting of the American Educational Research Association (AERA), Chicago, IL.
*Bulus, M. & Dong, N. (2015, April). Differential effectiveness of early childhood education programs on reducing reading achievement gaps. Paper presented at the 2015 Annual Meeting of the American Educational Research Association (AERA), Chicago, IL.
Dong, N., Reinke, W. M., Herman, K. C., Bradshaw, C. P., & Murray, D. W. (2015, March). Meaningful effect sizes, intra-class correlations, and proportions of variance explained by covariates for planning 3 level cluster randomized experiments in prevention science. In J. Spybrook (Chair), Improving the design of cluster randomized trials in education and prevention research. Symposium conducted at the Spring 2015 Research Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Dong, N. & Lipsey, M. W. (2014, November). How well propensity score methods approximate experiments using pretest and demographic information in educational research? Paper presented at the 35th Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Albuquerque, NM.
Dong, N. & Maynard, R. (2014, October). Applications of power analysis in the designs of multilevel experimental and quasi-experimental evaluations. Workshop presented at Evaluation 2014, the annual conference of the American Evaluation Association, Denver, CO.
Cravens, X. & Dong, N. (2014, April). Academic press in cultures of learning: An examination of PISA 2009. Paper presented at the 2014 Annual Meeting of the American Educational Research Association (AERA), Philadelphia, PA.
Dong, N. (2014, March). Power analysis to detect the effects of a continuous moderator in 2-level simple cluster randomized experiments. In J. Spybrook (Chair), Power analysis for moderator and mediator effects in clustered designs. Symposium conducted at the Spring 2014 Research Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Dong, N. & Lipsey, M. (2014, March). Empirical benchmarks of hidden bias in educational research: Implication for assessing how well propensity score methods approximate experiments and conducting sensitivity analysis. Paper presented at the Spring 2014 Research Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Reinke, W. M., Herman, K. C., & Dong, N. (2014, March). The Incredible Year Teacher Classroom Management Program: Initial findings from a group randomized control trial. Paper presented at the Spring 2014 Society for Research on Educational Effectiveness (SREE) Conference, Washington, DC.
Lipsey, M. W., Hofer, K. G., Dong, N., Farran, D. C., & Bilbrey, C. (2013, November). Effects of a public prekindergarten program on achievement and behavior: A randomized field experiment in Tennessee. Paper presented at the 34th Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Washington, DC.
Cravens, X. & Dong, N. (2013, April). School climate and student achievement: An examination of principal-teacher perspectives in TIMSS 2007. Paper presented at the 2013 Annual Meeting of the American Educational Research Association, San Francisco, California.
Dong, N. & Maynard, R. A. (2013, April). Applying generalized propensity score methods to estimate the effects of child care quality on child outcome. Paper presented at the 2013 Society for Research in Child Development (SRCD) Biennial Meeting, Seattle, Washington.
Dong, N. (2013, March). Incorporating sampling weights into propensity score analysis of survey data: Making causal inference to the sampled population. Paper presented at the Spring 2013 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Dong, N. & Fuhs, M. W. (2012, November). Persistence of kindergarten math achievement gaps over time: Probability and relative risk. Poster presented at the 34th Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Baltimore, MD.
Lipsey, M. W., Hofer, K. G., Farran, D. C., Bilbrey, C. & Dong, N. (2012, September). Effects of the Tennessee voluntary pre-kindergarten program on school readiness. Paper presented at the Fall 2012 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Cravens, X. & Dong, N. (2012, April). Is professional community an important learning condition for student achievement? Cross-national evidence from TIMSS 2003 and 2007. Paper presented at the 56th Annual Conference of the Comparative and International Education Society, San Juan, Puerto Rico.
Cravens, X. & Dong, N. (2012, April). School-wide professional community and mathematics achievement: Cross-national evidence from TIMSS 2003 and 2007. Paper presented at the 2012 Annual Meeting of the American Educational Research Association, Vancouver, British Columbia, Canada.
Dong, N. (2012, March). Causal moderation analysis using propensity score methods. Paper presented at the Spring 2012 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC. (ERIC #: ED530452).
Dong, N. & Maynard, R. A. (2012, March). PowerUp!: A tool for calculating minimum detectable effect size and sample size requirements for experimental and quasi-experimental designs. In J. Spybrook (Chair), Power analysis for clustered designs: A demonstration of three programs. Symposium conducted at the Spring 2012 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Lipsey, M. W., Farran, D. C., Hofer, K. G., Dong, N., & Bilbrey, C. (2012, March). Effects of the Tennessee voluntary pre-kindergarten program on school readiness. Paper presented at the Spring 2012 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Turner, K. A., Lipsey, M. W., Farran, D. C., Dong, N., Wilson, S. J., Fuhs, M. W., Vorhaus, E., & Meador, D. (2012, March). Academically relevant measures of executive function: Development and validation of assessments for preschool children. Paper presented at the Spring 2012 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Dong, N. & Hawkinson, L. (2011, November). The effects of program type on children’s school readiness: Main effects and variability of effects by conditions using propensity score analysis. Paper presented at the 33rd Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Washington, DC.
Cravens, X. & Dong, N. (2011, November). Learning-centered professional community and mathematics achievement: Cross-national evidence and progress in the United States. Paper presented at the 25th Annual Convention of the University Council for Educational Administration (UCEA), Pittsburgh, Pennsylvania.
Cravens, X., Fleming, P., Dong, N., Qian, H., Walker, A., Kwan, P., & Chen, P. (2011, November). School-wide professional community and international assessment: Seeking a deeper understanding beyond the east-west comparison. Session paper presented at the 25th Annual Convention of the University Council for Educational Administration (UCEA), Pittsburgh, Pennsylvania.
Dong, N. (2011, September). Using propensity score methods to approximate factorial experimental designs. Paper presented at the Fall 2011 Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC. (ERIC #: ED528839).
Dong, N. & Maynard, R. A., with Hofer, K. G. (2011, April). PowerUp!: A tool for calculating minimum detectable effect size for experimental and quasi-experimental designs. In S. Konstantopoulos (Chair), Design considerations for experimental studies in education. Symposium conducted at the 2011 Annual Meeting of the American Educational Research Association, New Orleans, Louisiana.
Dong, N. & Cravens, X. (2011, April). Identifying malleable school-level instructional conditions for mathematics achievement from TIMSS 1999 and 2007. Paper presented at the 2011 Annual Meeting of the American Educational Research Association, New Orleans, Louisiana.
Weathers, J. M., Hsu, H., Dong, N., & Meagher, S. (2011, April). A propensity score matching analysis of the effects of inclusive special education services. Paper presented at the 2011 Annual Meeting of the American Educational Research Association, New Orleans, Louisiana.
Huang, C., & Dong, N. (2011, March). Factor structures of the children’s depression inventory: A meta-analysis of pattern matrices. Paper presented at the 19th European Congress of Psychiatry, Vienna, Austria. Abstract published at European Psychiatry, 26, Supplement 1, 637. doi: 10.1016/S0924-9338(11)72343-4
Dong, N. & Lipsey, M. (2011, March). Biases in estimating the treatment effects due to attrition in cluster randomized controlled trials: A simulation study. Paper presented at the Spring 2011 Research Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC. (ERIC #: ED517992).
Lipsey, M., Farran, D., Hofer, K., Bilbrey, C., & Dong, N. (2011, March). The effects of the Tennessee voluntary pre-kindergarten program: Initial results. Paper presented at the Spring 2011 Research Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Dong, N. & Lipsey, M. (2010, November). Biases in estimating the treatment effects due to attrition in completely randomized experiments: A simulation study. Paper presented at the 32nd Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Boston, MA.
Dong, N. & Maynard, R. A. (2010, November). Child care quality and child outcomes: An exploratory threshold analysis plus a validation using propensity score matching method. Paper presented at the 32nd Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Boston, MA.
Griffen, A. S., Hawkinson, L., Dong, N., & Maynard, R. A. (2010, November). The effect of childcare subsidies on children’s cognitive development. Paper presented at the 32nd Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Boston, MA.
Cravens, X. & Dong, N. (2010, October). Leadership, learning-centered school conditions, and mathematics achievement: What can we learn from TIMSS? Paper presented at the 24th Annual Convention of the University Council for Educational Administration (UCEA), New Orleans, LA.
Dong, N. & Maynard, R. A. (2010, August). Selection and specification of the statistical models for analyzing longitudinal cluster randomized trials. Paper presented at The Second Biennial Meeting of The EARLI (European Association for Research on Learning and Instruction) Special Interest Group 18 – “Educational Effectiveness: Models, Methods and Applications”, Leuven, Belgium.
Dong, N. & Lipsey, M. (2010, March). The statistical power of the cluster randomized block design with matched pairs: A simulation study. Paper presented in the 2010 Annual Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC. (ERIC #: ED512728).
Mehta, N., Maynard, R. A., & Dong, N. (2010, March). The trade-off of bias versus power in regression discontinuity design. Paper presented in the 2010 Annual Conference of the Society for Research on Educational Effectiveness (SREE), Washington, DC.
Dong, N. & Maynard, R. A. (2009, April). The effects of Head Start: An evaluation using propensity score method and sensitivity analysis. Paper presented in the Annual Meeting of the American Educational Research Association, San Diego, California.
Dong, N. (2009, April). A spatial analysis of contextual and school effects on public elementary school performance in Philadelphia. Paper presented in the Annual Meeting of the American Educational Research Association, San Diego, California.
Dong, N. & Maynard, R. A. (2009, March). The effects of Head Start versus center-based care: Estimates for national samples matched using propensity scoring. Paper presented in the Annual Conference of the Society for Research on Educational Effectiveness (SREE), Crystal City, VA. (ERIC #: ED514946).
Dong, N. & Maynard, R., & (2009, March). Drawing causal inferences from a longitudinal cluster randomized experiment with crossovers: An evaluation of a distributed leadership program in urban schools. Paper presented in the Annual Conference of the Society for Research on Educational Effectiveness (SREE), Crystal City, VA. (ERIC #: ED524707).
Dong, N. (2008, May). The effects of Head Start on children’s kindergarten retention, reading and mathematics achievement in fall kindergarten: Comparing the evaluation results between direct OLS and propensity score method. Poster session presented in Mid-Atlantic Causal Inference Conference, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD.
Dong, N. (2008, March). Reinterpreting “Teach For America” evaluation results about teacher effects on student academic achievement. Paper presented in the Annual Meeting of the American Educational Research Association, New York.
Dong, N., Maynard, R. A., & Perez-Johnson, I. (2007, November). Issues in calculating average effect sizes in meta-analyses. Paper presented at the 29th Annual Association for Public Policy Analysis and Management (APPAM) Research Conference, Washington, DC.
Reports
Lipsey, M. W., Nesbitt, K. T., Farran, D. C., Dong, N., Fuhs, M. W., & Wilson, S. J. (2014). Learning-related cognitive self-regulation measures for prekindergarten children with predictive validity for academic achievement. Peabody Research Institute, Vanderbilt University.
Lipsey, M. W., Hofer, K. G., Dong, N., Farran, D. C., & Bilbrey, C. (2013). Evaluation of the Tennessee Voluntary Prekindergarten Program: Kindergarten and First Grade Follow-Up Results from the Randomized Control Design (Research Report). Nashville, TN: Vanderbilt University, Peabody Research Institute.
Hofer, K. G., Lipsey, M. W., Dong, N., & Farran, D. C. (2013). Results of the Early Math Project– Scale-Up Cross-Site Results (Research Report). Nashville, TN: Vanderbilt University, Peabody Research Institute.
Lipsey, M. W., Farran, D. C., Bilbrey, C., Hofer, K. G., & Dong, N. (2011). Initial results of the evaluation of the Tennessee voluntary pre-k program. Working Paper. Peabody Research Institute, Vanderbilt University.