Previous posts in our Data by Design series have explored the many ways that nonprofits can use their data to improve their decision making, enhance their collaboration, and set the stage for long term growth. But you cannot do any of these things without first ensuring the data you collect can be trusted. To speak with confidence about your organization’s accomplishments, you need to maintain strong data integrity.
When we say that data has integrity, we mean that it is complete, valid, and accurate.
- Complete data means that we have collected all the information we need. Of course, we will never be able to collect all the data we want, but if the data we are collecting is essential to our work (and it should be!), efforts should be made to make it as complete as possible.
- Valid data means that the information is what it is supposed to be. If the entry for “First Name” says “19,” the data is probably not valid.
- Accurate data is by far the hardest to verify, and it can change. Data that is complete, valid, and accurate today will still be complete and valid tomorrow, but it may no longer be accurate.
Maintaining data integrity is a process, and the form that process will take depends on what data is collected, how it is obtained, and what will be done with it. Nevertheless, there are a few commonalities to all adequate data integrity processes.
- Start at the end
Before you even think about what data to collect, think about what you are hoping to achieve with it. What reports would you like to generate? What information do you want your case managers to have?It is tempting to collect everything and figure out what to do with it later, but the more data points you obtain, the harder it will be to maintain data integrity. You should have a good reason for every piece of data you collect and not just for privacy reasons. Obtaining only what you need will make it far easier to maintain data integrity and to notice when it is absent.This rule is especially true when the data is being collected and entered manually. The fewer data points your users need to collect, the more likely they are to do it right.
- Raise awareness
When you train users to collect data, make sure they understand how important it is that they do it right. It might not always be obvious why a particular data point is important. It’s essential to make it clear that your impact, and possibly funding, depend on doing it right.Staff should take time to audit incoming data and call attention to incomplete, invalid, or inaccurate data. This will further highlight the importance of good data collection.
- Leverage your technology
One of the simplest ways to assure data integrity is to use a data collection tool that is capable of validating the data as you enter it. In the simplest case, this might mean not allowing a record to save until specific fields have been filled or making sure that the entry for “Birthdate” is a date. More complex validation might involve making sure that a participant added to a program has a participant record and that it meets criteria for the program (e.g. “under 18,” “female”).One important caveat here is that if your data validation rules are not well-considered, they can increase the chances of corrupt data. If your data validation insists that “Employer Name” be filled, data collectors will sometimes need to enter garbage data to proceed. For particularly essential data points, you may even leverage your database’s reporting tools to help you audit the data regularly. It is worth spending effort to get these validation rules correct because cleaning your data after the fact will be far more time-consuming and maybe even impossible.
Maintaining data integrity is not the most exciting, but the good news is it’s also not very hard. It just takes some planning, a little awareness, and some streamlined validation tools. And in return, you get to speak with confidence about the incredible work you and your organization are doing.
At Social Solutions, we think about these data challenges every day. As we continue to improve our products, we strive to make the data entry process faster, easier, and cleaner. We want to simplify how our clients use their data, not just to reflect on what they have done, but to inform their decision making in real-time as they help their participants.
And it all starts with data integrity.
About the Author
Kevin Burnham is a data analyst with over five years of experience gathering, auditing, cleaning, shaping, summarizing, plotting and modeling data. Before joining Social Solutions his experiences include teaching Arabic and modeling season ticket sales for NFL, NBA, MLS and NHL teams.