Why Data Quality and Data Cleansing Projects Fail
Data Quality and Cleansing initiatives are essential to improving overall operational and IT effectiveness. However many efforts do not get off the ground and get stalled before they really start. Read more
A Healthcare and Education Case Study
Hello Friends,
Please see the attached case study about how West Virginia University used DataMatch to save thousands of work hours and improve patient care.
Case Study
Title-Tracking Patient Records Across Multiple Databases
Industry: Health Care, Education
Situation: West Virginia University was tasked with assessing the long term impacts of certain medical conditions over time. Specifically if previous conditions affected long term health and patient care. The difficulty was that the databases records identifying the medical condition were in a separate system from the current health records provided by the state. Linking these records manually, which number in the hundreds of thousands, was a very time consuming process and threatened to derail research activities.
Solution: Using Datamatch, Data Ladder’s flagship product for data cleansing, WVU was able to clean records from multiple system and create a unified view of the patient over time. With the best in class data cleansing and matching routines, along with the included customized training, WVU was able to see unified results quickly and easily.
Results: With a unified view research was able to resume at a much quicker and more efficient pace. The cleansed data has been used and referenced in several medical journals, with the hope of improving patient care effectively and efficiently. With this success WVU is expanding the use of DataMatch across several other functions within the university.
More Than One Way to Skin a Database
I have been a solution manager for Data Ladder for several years and have been fortunate enough to learn more and new ways to use DataMatch 2011 to solve numerous data problems that I never even knew existed. From working with Hospitals analyzing critical data and preventing future chronic illnesses` to your every day saving cash by eliminating duplicates in a post mail campaign, DataMatch 2011 provides a wide array of intergrated tools that allow you to do much more than you would expect.
DataMatch 2011 provides you options to find and eliminate hard to find duplicates but also gives our clients many other cures for common database problems. From getting rid of unwanted characters across thousands and thousands of records to taking a column of data and parsing it out based on specific needs and sending that data to new or existing fields. DataMatch 2011 packs an incredible punch and should be investigated carefully to get the most out of this powerful world class software.
I personally encourage every possible user to ask our sales reps any possible questions that may come to mind or any possible scenario the user may have when it come to specific needs. You may be surprised how many different solutions DataMatch 2011 can provide you.
http://www.DataLadder.com/download.php
Manuel Suarez
Sales Manager
866-557-8102
tsuarez@DataLadder.com
Data cleansing techniques
The amount of data we all deal with every day is expanding rapidly. With expanding data and the ongoing addition of data sets, keeping data clean is essential. There are several different simple data cleansing techniques that can avoid and correct data quality issues. All of the following are included in our DataMatch product that we can walk you through in a customized WebEx demonstration.
Data Cleansing Technique 1: Data Profiling
Know what you have in your data. A simple look at the min/max, top values, and data types in every column/field of your data can flag data quality issues or misunderstandings within the data set.
Data Cleansing Technique 2: Simple Data Cleaning
Sometimes there are simple changes that go a long way. Removing a space, changing all O’s to zeroes, making a copy of a field to manipulate later, etc. Additionally other simple functionality like recognizing that Jon is a nickname for Jonathan
Data Cleansing Technique 3: Standardization and Parsing
Sometimes data is entered in an uncontrolled manner resulting in pieces of data in the wrong place. The zip code in the city field, etc. DataMatch is equipped with advanced libraries and pattern recognition to find and parse out the most common standard address pieces. Additionally other simple functionality like recognizing that Jon is a nickname for Jonathan and is a Male gender name can be very helpful for cleaning your data and making it more useable.
For non standard information our Wordsmith and Regular Expression creator allows for an infinite number of customized parsing possibilities.
Data Cleansing Technique 4: Duplicates and Fuzzy Matching
Simple misspellings are very common, Somewhere Way and Somwhere Way both look the same to a person, but to a machine they are different. DataMatch’s fuzzy logic algorithm can detect these subtle differences quickly and combine the records, either to simply flag as a duplicates, help determine which record should be a master complete record, or just to transfer data between the records as you see fit.
Our standardization and parsing logic allows you to create matches on parsed out text, like street number, zip code, etc. Additionally you can create multiple definitions of what a match is. For instance you can say any records with the same email address are a match, and any records with similar street, person, and city names are also a match.
There are a lot of details to the above data cleansing techniques and we hope you will contact us so we can show you how DataMatch can meet your data cleansing needs with a demonstration on your own data and specific needs. Phone: 866-557-8102 Email: Sales@DataLadder.com
Data Ladder LLC, a Provider of Simple and Affordable Data Cleansing Software, Announces the Release of DataMatch 2011
Data Ladder LLC, a provider of simple and affordable data cleansing software, announces the release of DataMatch 2011. With ease of use design changes, additional cleansing functionality, and best in class fast fuzzy matching algorithms, DataMatch 2011 brings the capability of a strong data cleansing software to the desktop. DataMatch 2011 provides key capabilities to keep your databases and lists clean and free of duplicates, saving time and money for business professionals.
Free Trial Available at http://www.dataladder.com/download.php
“The Data Warehousing Institute (TDWI) estimated that inaccurate customer data costs businesses (over) $611 billion a year in postage, printing, and staff overhead. Frighteningly, the real cost of bad data is higher. Data problems can alienate customers, create revenue and cost leaks, undermine process efficiency, delay expensive projects, and expose an organization to compliance risks. In short, bad data can make it hard for the business to achieve its financial and strategic goals.”
Key New Features:
Intuitive wizard sets data cleansing options based on 4 questions, drastically reducing data cleansing setup time and confusion. Additionally Data Ladder’s world class customer support is available to walk every customer through their own data cleansing project.
A simplified user interface allowing visibility on how each data cleansing step has changed the data set, ability to undo changes, and the ability to save your work as a project that can be reused and scheduled.
Visually appealing data profiling and match results reports that can be exported to Excel and PDF.
Improved proprietary matching algorithms providing best in class matching accuracy and speed.
Data Profiling: including automated field type identification and extraction (Address, First Name, etc.)
Multiple Match Definitions: Automatically identifies multiple match definitions within your data set. Examples: Simultaneously identify and group all records with matching emails, at the same time identify and group all records with similar person and street names in the same zip code.
Multiple deduplication options
Survivorship rules for defining what record or field should remain in the cleansed data set. Example: Keep all records in a matched group with the newest date, and merge all non null emails. Record level traceability of all changes.
Miscellaneous new features; column merge, regular expression builder, etc.
With additional features and knowledgeable customer support, Data Ladder’s DataMatch 2011 provides a simple, world class data quality suite to the business user at an affordable price. For more information please visit http://www.DataLadder.com.
About Data Ladder
Data Ladder is a data quality and cleansing company dedicated to helping you “Get the Most Out of Your Data” through Data Matching, Profiling, Deduplication, and Enrichment.
We strive to keep things simple and understandable in our product offerings to give our customers the best solution and customer service at an excellent price.
Our products are in use across the Fortune 500 and we are proud of our reputation of listening to our customers and rapidly improving our products.
Remove Duplicate Records
Duplicate Records Impact on Sales and Operations
Quick post.
The major issues with duplicate records are often not felt by IT departments.
Typically the every day pain is felt within the heart and soul of a company, it’s sales force and operations. Sales sees firsthand the cost and embarrassment of duplicate customer contacts, mailings, and wasted time trying to decipher which customer number to use for the same customer when multiples exist in the system.
Operations sees the returns, customer complaints, and duplicated effort.
While good IT departments are right on top of the issue, sometimes a reminder is needed, or even the opportunity for sales and operations to take the matter into their own hands with a simple and easy to use software suite like DataMatch.
IT is especially important in realizing the issues that duplicate records have throughout a company. A duplicate customer master can impact the whole company, confusing auditors, new employees, and generally aggravating management from a mistake and reporting perspective.
Now is the time to fix the issue once and for all, give us a call at 866-557-8102 for a free consultation and walk through of our data quality solutions.
The Biggest Issue with Data Quality
Many realize that data quality is an issue and wish to address it somehow. However many organizations struggle with the same issue: The need for a simple plan.
A Simple Plan to show success in one week
Data quality initiatives can be complex and intimidating. Most vendors offer only large scale, expensive, and time consuming solutions that are not guaranteed successes based on previous engagements. Some organizations develop initiative fatigue after a few months of constructing elaborate schemas and structures with an outside vendor. Even the vendor screening process can be a barrier to starting a data quality initiatives.
While each plan may differ, we find the following quick start plan gets things moving immediately and maintains the morale of the team with fast successes and demonstrated ROI.
Step 1: Where is the biggest pain?
Maybe it was a dissatisfied customer, or an awareness that marketing campaigns are duplicating effort, or many separate systems/spreadsheet/lists causing frustration. Chance are there is one large issue that prompted the search for a data quality solution. This can be the rallying point for the organization.
Be careful not to complicate matters with too many technical terms or adding to the scope of the project at this point.
Step 2: Someone to talk to.
Most data quality problems are similar and chances are someone has encountered your problem before. A quick discussion can save a lot of time, help avoid mistakes, and motivate the team by showing that success is very likely. Due to the nature of our software we have encountered many different data quality issues on all different types of data. Feel free to contact us and we’ll schedule a personalized WebEx discussing your specific problem and how we can help. Contact info: Sales@DataLadder.com Telephone: 866-557-8102
Step 3: Simple, affordable, and customizable solution
Intuitive easy to learn solutions are paramount. Our DataMatch software comes with walkthroughs, video demonstrations, and a customized WebEx session. Users are up and running in an hour. It is also important to have a solution that is customizable as no 2 data quality situations are exactly the same. May be you want to remove duplicates from a customer list. But what you think of as a duplicate may change (same address, same phone number, or maybe the company name)
Step 4: Celebrate your success
Once the data set has been cleaned, let everyone know about your success. This improves morale and demonstrates the ability to address data quality issues simply, quickly, and affordability. Now is the time to determine what the next issue is on the data quality agenda, and discuss ways to keep data quality high at the start of each process.
Please let us know your thoughts on the simple plan, and any other topics you would like addressed in this blog.
Every data cleaning situation is unique
A quick post today.
One often overlooked fact is that every data cleansing situation is very unique.
Take a simple customer deduplication (removal of duplicates) exercise. At first glance it is a very simple problem. Identify the duplicates, and remove them. However once you get into the details you realize there are several items worth considering.
1. How do you identify a duplicate? Is it the company name? Contact name? Address? Maybe you deal with 2 completely different offices that are the same customer (IBM in Australia and in the UK for instance)
2. Do you want to remove all information about a duplicate contact? There may be important contact information or customer notes associated with the record.
3. Have all affected stakeholders in your organization been made aware that the cleanup was occurring? There may be individuals and departments inside your own organization who should be notified to insure no unintended consequences occur.
4. Are there any new standards you’d like to apply? Capitalizing street suffixes, separating full name fields to a First and Last name field, etc.
Note that Data Ladder is here to walk you through these issues which is why we give free personalized WebEx demonstrations addressing your specific data cleansing activity.
Any other big questions that I missed? Feel free to comment below. We welcome and thank you for taking part in the conversation.
Why Email Cleaning is Crucial to Effective Correspondence
Email has grown from a mere messaging medium to something indispensable in the corporate arena. Especially, official communication in the corporate sector happens through an organization’s mailing system. Though everything, from important decision-making to strategy-planning, happen in a matter of clicking send/receive/forward from your desktop, maintaining an effective email system is not as easy as it sounds. Read more
The Importance Of A Thoughtful Merge Purge Strategy
The importance of a thoughtful merge purge strategy
Merge Purge is the process of combining two or more lists or files, simultaneously identifying and/or combining duplicates and eliminating (Purging) unwanted records. The purpose of merge/purge is to clean the underlying data set to achieve productivity improvements, save on duplicate mailings, and increase customer satisfaction.
Read more
