When coaching and inference knowledge come from completely different sources
- Introduction
- Enabling Information Assortment
- Setting a Baseline
- Detecting Outliers
- Abstract
- References
This text is meant for knowledge scientists who’re both starting or wish to enhance their present knowledge validation course of, serving as a basic define with some examples. First, I wish to outline knowledge validation right here as it could have completely different meanings for different, related job roles. For the aim of this text, we’ll say that knowledge validation is the method of making certain the coaching knowledge used in your mannequin matches or is in step with inference knowledge. For some firms and a few use circumstances, you’ll not want to fret about this situation if the information is coming from the identical supply. Due to this fact, this course of should happen and is just helpful when knowledge is coming from completely different sources. A few of the explanation why knowledge wouldn’t be coming from the identical supply is that if your coaching knowledge is historic and custom-made (ex: options derived from current knowledge), and/or your inference knowledge is coming from dwell tables the place the coaching is snapshot knowledge. All that to say, there are many causes for this mismatch to be current and it will likely be extremely helpful to provide you with a course of at scale to make sure the information you might be feeding your mannequin at inference is what you — aka the skilled mannequin knowledge expects.
There are many methods you may allow knowledge assortment. However as soon as once more, first, we wish to outline the knowledge that’s collected, which might be the inference knowledge. We anticipate to have our coaching knowledge (composed of each practice and take a look at splits) already positioned someplace, maybe in S3, a file storage device, in a short lived desk in a database, even a CSV file, and so forth.
