Data & Analytics

Data-Driven Enterprise off Your Beat - John Duchneskie - Murfreesboro, TN - Sept. 30-Oct. 1, 2016

This handout accompanies a presentation of the same name by John Duchneskie, assistant managing editor at The Philadelphia Inquirer, at NewsTrain in Murfreesboro, Tennessee, on Sept. 30-Oct. 1, 2016. It includes why and where to learn data-journalism skills and where to find data, including specific databases for different beats. An additional handout, "Step-by-Step Guide to Data Exercises," also goes with the presentation. NewsTrain is a training initiative of Associated Press Media Editors. More info:
of 6
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  • 1. Page 1 of 6 Data-driven enterprise off your beat John Duchneskie | @jduchneskie | Why should I learn this?  Have you read anything from,, or The Upshot ( Thought so. Big data is all around us, and there are journalists and websites that are using it to break and illuminate news.  Yes, journalism jobs can be tough to find, and tougher to hold onto. Jobs for journalists who can handle data, however, are in demand. For proof, look at the job-posting board from the NICAR (National Institute for Computer-Assisted Reporting) conferences in 2015 ( and 2016 (  Having the ability to find and interpret data will help you discover stories you otherwise would not have found, bring insight and perspective to your work, and give you a competitive edge. What skills do I need to learn?  You need to be comfortable in a spreadsheet program such as Microsoft Excel. If you don’t have Excel, you can use the free Google Sheets. You will need to learn some basic functions, such as sorting, filtering and performing basic calculations. Definitely learn how to calculate percentage changes: it’s the new number minus the base number, divided by the base number. And where can I learn them?  Steve Doig, a Pulitzer Prize-winning journalism professor at Arizona State University, has this brief tutorial ( on the basics of Excel.  Kate Willson has produced these four videos ( to teach Excel basics to journalists.  The Global Investigative Journalism Network has this good, quick tutorial ( with 12 tips on getting starting in data journalism.  Once you have the basics down, Mary Jo Webster has compiled an incredibly comprehensive list of Excel tips and tricks called Excel Magic (  For larger datasets, you may want to learn how to use a database program, such as Microsoft Access. I find that I can almost all of my data wrangling in Excel, but every now and then I will turn to Access for the heavy lifting.  Computer-assisted reporting boot camps through Investigative Reporters & Editors:
  • 2. Page 2 of 6 Fellowships available: scholarships  While we’re at it, join Investigative Reporters & Editors (IRE): $70 a year/$25 for students. 3,500 tip sheets; 30 low-cost, cleaned-up government databases; 25,000 stories, NICAR-Learn short-video training for $25 for members/$40 non-members. Where can I find data?  Google it. Use Google’s Advanced Search ( functions to restrict your search to a specific domain (such as and choose .xls in the file type.  Find the hidden tech guy or gal in the basement. If you need data from a specific agency, try Googling for someone there with “data,” “IT,” “GIS” or other similar techie term in their title, and then ask that person. I saved the attendee list for a local tech conference I attended and have found that to be a great resource for identifying the person who has the goods.  Check records-retention schedules. Most government entities have a records-retention schedule that outlines what records they have, and how long they have to be kept. It can be a nice index to what’s available.  See what others are up to. is a great resource for data stories. Many will tip you off to interesting data sets that you can obtain on your own.  Join local technology and open-data meetups, such as Hacks and Hackers or Code for America, and meet people who have a passion for using technology to improve society (and maybe your stories).  See if there is an Open Data portal for your government. For example, check out or The federal government has created A list of more open-data portals is here. (  FOIA it. If your government agencies brush off your data request, you can file a formal Freedom of Information Act request. Follow up, and be prepared to wait.  NICAR Data Library ( Access national databases or just statewide slices -- such as the National Bridge Inventory (is the bridge in your town structurally deficient or functionally obsolete?) -- that have been cleaned and are ready to use, for a small fee.  NICAR-L mailing list ( The data-journalism community has a deserved reputation for helping others. Subscribe to this list, ask questions, and join the conversation.  Follow data journalists on Twitter. Look at what others in the field are doing and thinking and get inspired. For starters, here is a public list of data journalists on Twitter ( from Martin Stabe at the Financial Times.
  • 3. Page 3 of 6  Data Is Plural. Subscribe to this “weekly newsletter of useful/curious datasets” ( for a peek at the usual and the unusual.  Read annual reports, board reports, and the like. The totals and averages that are contained in regular reports don’t come from thin air. Asking where the numbers came from can tip you off to the existence of important, unadvertised data sets.  Make your own. Sometimes you have to make a data set yourself. Consult with experts to make sure you’re not doing anything unwise. Other useful data sources  U.S. Census Bureau. Just about anything you want to know about people and society is here, continuously collected, down to your town, ZIP code and census block. The amount of data that is collected can seem overwhelming to new users, but taking the time to familiarize yourself with census data will be richly rewarding. Start with American FactFinder, ( the main gateway to census data.  On the Map ( Another product from the Census Bureau that tracks the relationship of where people live and where they work. The Philadelphia Inquirer used On the Map data to produce this graphic for print ( and this interactive map ( to show the jobs and commuting patterns in our region.  The real estate website provides data over time ( on home prices and rental values down to the ZIP code or city neighborhood.  Find data, tools and graphics covering a wide range of issues.  FEC Itemizer ( This data tool from ProPublica allows you to easily browse electronic campaign-finance filings from the Federal Election Commission. You can also get this data straight from the FEC here (  Bureau of Labor Statistics ( Track data for inflation, jobs, pay, workplace injuries and more. Try localizing the data to compare trends over time, such as the job growth in your state versus neighboring states.  FRED Economic Data ( The Federal Reserve Bank of St. Louis has created a data portal where you can download and graph a huge array of economic data.  CDC. The Centers for Disease Control and Prevention ( has statistics on a wide variety of health topics, and links to tools and other resources. It has data on how every single American dies. What are the highest rates in your area? Why? The County Health Rankings ( website lets you map and compare health factors by county.
  • 4. Page 4 of 6 So what else can you do with data on your beat? Compiled by Matt Wynn at the Omaha World-Herald and Jill Riepenhoff at The Columbus Dispatch and Linda Austin at APME’s NewsTrain Government - Tax assessments. Gold mine of stories that people want to read. Potential graft or quid pro quo. - Salaries. Topline numbers can be predictable and boring. But breakdowns -- overtime, special incentive pay, etc. -- can offer a new layer worth exploring. - Tax scofflaws. People and businesses that don’t pay up often make their own special list. Worth getting. - Budgets. The more detailed, the better. Changes over time can be especially interesting. - Licenses. Any licenses awarded by your agency can be telling. - Inspections. Restaurants, weights and measures, cleanliness, safety, etc. - Purchase records. What’s getting purchased, from whom and for how much? - P Cards. Similar to purchase records, but often with less oversight. Get credit-card statements for cardholders within the agency. - Campaign finance. Who’s giving? Who’s getting? When? ( and for federal candidates and for state candidates and for spending on political ads at local broadcast stations) Education - Test scores. Over time, by race and poverty or special education status. - Campus crime. Compiled by IRE and available for cheap, breaks down crimes on every college campus in the US. Cross-checked with police records, can lead to some valuable results. ( - Teacher rosters. Which schools get the greenest teachers? Which has the most experienced, the most educated? - Repairs, maintenance and repair requests. Maintaining so many buildings and so much equipment costs money. How is your district keeping up, and is every school treated fairly? - College Scorecard. IRE has simplified the College Scorecard database released by the Obama administration in 2016 and made it free to its members. The database tracks who gets into what school and what they do after graduation for 7,800+ campuses. ( - National Center for Education Statistics. Has a wealth of data including the Public Elementary/Secondary School Universe Survey Data (, Local Education Agency (School District) Universe Survey Data ( and the Local Education Agency (School District) Finance (F-33) Data ( - Poverty by school district. The Census Bureau tracks income and poverty within school districts. (
  • 5. Page 5 of 6 - Department of Education’s Office for Civil Rights Data Collection. Contains data on equity of programs and discipline by school and district. ( Cops and courts - Sex-offender registration. Do they live near schools, day cares? Are they all living in the same area? Are the registration records even accurate? - Crime logs. Where are burglaries most likely? Is there a time of year that’s most dangerous? If your department has data with narratives, all the better. Can be compared to FBI Uniform Crime Reporting (UCR) data to see if reports are accurate. - Jail/prison logs. Who’s been in longest? Who’s been in most often? - Police discipline. - Court records. Which judge is harshest? How does your county court differ from those around the state in terms of sentencing? Which attorney pleas down the most? - National Crime Victimization Survey. The Bureau of Justice Statistics surveys a national sample of nearly 160,000 on whether they have recently been a victim of actual or attempted rape, robbery, assault, burglary, larceny or motor-vehicle theft. Data include demographics of the victim and time and place of occurrence. ( Health - Vaccination rates - Various inspections and complaints. Nursing homes, hospitals, home health-care agencies. - CMS (Centers for Medicare & Medicaid Services) data sets. The feds have interesting data grading and comparing hospitals, nursing homes and the like along a variety of measures. For example: Sports - Minor league baseball: - College athletic department salaries. See USA Today for top coaches’ salaries in football: - and basketball: - NCAA research data: - Major NCAA infractions: - Academic-progress rates for college athletes: - High school sports-participation rates: Grab bag - Potholes. Cities should have databases reflecting pothole locations, the date they were notified of the issue and the day it was fixed. Comparing all those issues can show you hard-hit areas of town, how well your city
  • 6. Page 6 of 6 responds in general, or if services differ depending on where in the city a pothole is located. - Lawsuits and claims against the city. Pretty much just potholes, part two. Can let you see if potholes are getting more severe, based on the number of claims or actual value paid out. Might also indicate other persistent issues in your city or its infrastructure. - Pet names: Many local governments track this when issuing dog tags; if not, the humane society or others may have the info. You can look at the most popular name, the most popular name by breed or by type of pet. You can break down into ZIP codes or cities, if you have a large enough coverage area. You can also look at most unique names. - Gas-pump inspections: An excellent first CAR story. The state department of weights and measures tests every gas pump to see whether, when it reports pumping a gallon, it indeed does just that. Those data are recorded and available for a nominal fee. You can see the worst pump in your town, how your town compares to others and so on. - Google Public Data Explorer: Google has collected public data sets and offers free, embeddable charts. - Has a searchable database of 10 billion visualizations, including many that are off the national/international news. They are embeddable for free. Minimal customization is available. You can sign up for email alerts for new visualizations on your beat. More inspiration  60 Data-Driven Ideas in 60 Minutes ( presentation by Jodi Upton and Mary Jo Webster.  Journalist’s Toolbox. Mike Reilley has compiled this very comprehensive list of data visualization and online journalism ( links.  Diving into Data Journalism ( An overview of the rise of data reporting, tips on how to get started in data journalism, and a list of resources.  4 kinds of stories you are doing wrong without data:  How to avoid 10 common mistakes in data reporting:  Essential tips and tools for beginning data journalists: More questions? There’s nothing I like more than helping other journalists. Here’s how to reach me: John Duchneskie | The Philadelphia Inquirer | 215-854-5712 | | @jduchneskie
  • Search
    Similar documents
    View more...
    Related Search
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks

    We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

    More details...

    Sign Now!

    We are very appreciated for your Prompt Action!