Workflow of Data Analysis: A Short Course

This seminar is for researchers who are trying to establish or improve their workflow. I do not expect participants to be expert programmers; this seminar should be accessible to very novice Stata and R users, while still being useful to more advanced users. Lessons from this seminar balance ease of use with proper functioning, introducing researchers to useful tools, e.g., dual-pane browsers, macro programs, plain text editors, etc. For those who are already familiar with these tools, this seminar will teach you how to optimize them. Lessons from this seminar should make conducting research less painful, more efficient, more accurate, and reproducible.

This is a hands-on seminar with ample opportunities to plan and practice your workflow.

Some highlights include:

• Planning (analyses, sensitivity analyses, variable construction, etc.)
• Organizing files using a standardized directory structure
• Preserving data and findings
• Effectively documenting findings, data sources, cleaning methods
• Separating data management and analyses using dual workflow
• Writing robust script files
• Naming variables
• Labeling variables and values
• Creating research that is both reproducible and replicable
• Examining data quality

The empirical examples and exercises in this course will emphasize Stata and R. To fully benefit from the course, you should use your own computer with a recent version of Stata or R installed.

Stata users are encouraged to use Stata version 18, but earlier versions should also work for most exercises.

R users should also download and install RStudio, a front-end for R that makes it easier to work with. This software is free and available for Windows, Mac, and Linux platforms.

Seminar participants who are not yet ready to purchase Stata could take advantage of StataCorp’s free 30-day evaluation offer or their 30-day software return policy.

If you’d like to use R for this course but don’t yet have much experience with that package, here are some excellent on-line resources for building your R skills.

This course is for anyone who wants to improve the efficiency and accuracy of their data management, analysis, and presentation. You should have experience with data analysis, as well as familiarity with Stata or R.

PART 1: INTRODUCTION TO WORKFLOW

1. What is “workflow”?
2. Why care about WF?
3. WF and replication
4. Steps in and principles of WF

PART 2: PLAN, ORGANIZE, DOCUMENT, AND PRESERVE

1. Planning research projects in the:
   a. Large (overall questions, project checklist, and timeline)
   b. Middle (data cleaning, analyses, tables, and figures)
   c. Small (naming variables, naming files, value labels, and order of
   analyses/cleaning)
2. Organizing files and folders
3. Documentation
4. Preserving data and preventing loss
5. Replication

PART 3: SCRIPT FILES IN R

1. Strengths and weaknesses of R for workflow
2. Dual workflow
3. Robust script files
4. Legible script files
5. Automation in script files

PART 4: CLEANING, LABELING, & MISSING DATA

1. Naming and labeling variables
2. Missing data
3. Merging data
4. Verifying data

PART 5: ANALYZING & PRESENTING FINDINGS

1. Principles of data analysis
2. Documenting provenance
3. The posting principle
4. Presenting findings

PART 6: COLLABORATION

1. Key factors in collaboration
2. Introducing workflow with co-authors
3. Coordinating workflow with multiple authors

“This should be required learning for all researchers. The course covers the principles/rationale for having good workflow as well as concrete steps you can do right away to improve your own workflow. I was able to immediately implement steps that I know will make me a better researcher. I also have the resources I need to tackle larger changes to my workflow in the coming weeks. It’s been 5 years since I completed my PhD and I wish I had taken this course in my first year. I left this course feeling excited and empowered.”
  Marta Mulawa, Duke University

“This was one of the most practical and useful Statistical Horizon workshops I’ve participated in to date! Bianca’s teaching style was great – she was knowledgeable and provided a nice blend of content coverage as well as applied exercises.”
  Sage Hawn, Boston University/VA Boston Healthcare System

“I loved the big-picture approach! The way the instructor merged together all aspects of workflow from beginning to end so that everyone would find something to help them improve. I appreciated the practical advice to start with a few changes and then add in more as time goes on. Very inspiring!”
  Diane Holmberg, Department of Psychology, Acadia University

“Great class! Bianca Manago is a superb teacher; positive, enthusiastic, extremely knowledgeable, clear, and responsive and helpful to students. I gained extremely valuable insights, principles, tips, and practical strategies that will improve the quality, reproducibility, and replicability of all my research.”
  Ken Coburn, Health Quality Partners (HQP)

“Bianca was a great, knowledgeable entertaining teacher.  She really made what could have been very dry material seem exciting. The knowledge I gained in this course will revolutionize the way I approach my research projects.”
  Laura Prichett, Johns Hopkins 

“Bianca’s depth of material and her engaging delivery was wonderful. The course was very practical in perspective. I have attended other R courses that did not increase my working knowledge – Bianca’s workshop has already. I also liked the opportunities for useful student interaction, which I associate with Bianca’s teaching style.”
  Rod Ling, University of Newcastle