{"id":1554,"date":"2023-09-05T23:49:26","date_gmt":"2023-09-05T23:49:26","guid":{"rendered":"https:\/\/fabiogori.ai\/?p=1554"},"modified":"2023-09-29T13:10:05","modified_gmt":"2023-09-29T13:10:05","slug":"the-most-important-yet-answerable-question-in-ai","status":"publish","type":"post","link":"https:\/\/fabiogori.ai\/it\/allblogs\/diversely-intelligent\/the-most-important-yet-answerable-question-in-ai\/","title":{"rendered":"The most important yet unanswerable question in AI"},"content":{"rendered":"

<\/p>\n\n\n\n

The most important yet unanswerable question in AI<\/strong><\/h2><\/div>\n\n\n\n

The uncomfortable truth about how much data you need for using AI<\/em><\/h4>\n<\/div>\n\n\n\n
\"\"<\/figure>\n\n\n\n

Photo by Joshua Sortino<\/a> on Unsplash<\/a><\/p>\n\n\n\n

Data has a central role in the development of AI and ML models. These tools are trained on data; without data, there is no AI model. You may see this model training as a way to extract value from data; this suggests that, in many situations, having more data brings more value, that means better performances of our AI and ML algorithms.<\/p>\n<\/div>\n\n\n\n

\"\"
Performance of AI and ML models usually increases with the quantity of data used for training<\/em><\/figcaption><\/figure>\n\n\n\n

However, collecting data, especially individual data points with specific label information, can be a challenge in many non-digital sectors like manufacturing and life sciences.<\/p>\n

While e-commerce and digital marketing can easily collect millions of data points from users and people browsing the internet, things are very different when behind a data point there is a real-life object or sample. For instance, in molecular biology, omics technologies can collect many variables, but often the sample sizes is limited due to efforts required for setting up the experiments.\nIn medicine, many studies might only involve a handful of patients because of the\nchallenges associated with patient recruitment<\/a>.<\/p>\n

Manufacturing data comes with its own set of challenges: a new product might have no data, while existing products might suffer from rare anomalies, lack of label information, or missing measurements of crucial variables (e.g. what consumable substances are used in every shift, the fact if operators change or not, manual interventions, etc.).<\/p>\n

Given the challenges in collecting data, before embarking in an AI project, it is very sensible to estimate how many data needs to be collected before an AI model can perform reasonably well. In domains where data is scarce and costly to gather, estimating the necessary sample size becomes crucial for budgeting.<\/p>\n

However, answering the question \u201cHow much data we need for our AI project?\u201d is not possible.\nA rule of thumb suggests that the collected data samples should be 10 times the number of variables. But this is merely a guideline, and we have already explained that the number of variables used could be wrong, because the collected variables could be missing very important ones and including irrelevant ones. Since data science is about answering questions based on data, we face the chicken-and-egg causality dilemma: we need data to determine if we have enough data for our goals.<\/p>\n

If we do not know how much data we need for our AI project, how can we proceed? We recommend a data-centric approach<\/a>, where AI performance improvement are obtained not primarily by changing the model but by changing the data; in our case, it means collecting the data iteratively:<\/p>\n

    \n
  1. Based on previous data collection, and om the best of our knowledge, determine all informative variables we can measure in the upcoming data collection<\/li>\n
  2. Collect new data; the number of samples depend on budget and goals<\/li>\n
  3. Process both old and new data to create a dataset that best represents real-world scenarios<\/li>\n
  4. On this newly-derived dataset, retrain an existing AI model or train a new AI model<\/li>\n
  5. Deploy the trained AI model in production<\/li>\n
  6. Measure regularly the performance of the AI model in production by using various metrics; if the measurements are unsatisfactory or start degrading, go back to point 1.<\/li>\n<\/ol>\n<\/div>\n\n\n\n
    \"\"
    The six steps of the iterative data collection process<\/em><\/figcaption><\/figure>\n\n\n\n

    This iterative approach offers several benefits. As mentioned in the iteration point 2, by collecting data step by step, you can adjust and guide the collection process to make your dataset distribution closely resemble real-world distribution; you can collect data about cases for which you are missing data, or realize you should collect additional variables. For example, in medical settings you may realize that the demographics of your patients underrepresent some categories, or in an anomaly detection projects for manufacturing you are gathering data at times when anomalies are least likely.<\/p>\n

    But there is also an additional reason in favor of this iterative approach: counteracting the so called data drift<\/a>. In many scenarios, unfortunately, older data becomes less and less representative of the present situation, so it is necessary to periodically retrain the AI model only on relatively recent data. By collecting data incrementally rather than in one go, we can better counteract the effects of the data drift: we can still make use of some of the data already collected. If all data were collected at once, the data drift would impact sooner and harder: the whole training data, and not just the oldest slices like in the iterative approach, would become obsolete.<\/p>\n

    In conclusion, AI needs data but it\u2019s challenging to estimate how much data are needed. To save on budget, and to maintain good performances, it is advisable to collect new data regularly and monitor the performances of the AI model.<\/p>\n<\/div>\n\n\n\n

    Fai gratuitamente il Data Maturity Quiz<\/a><\/strong><\/p>\n

    Nel mondo della data science, capire a che punto siete \u00e8 il primo passo verso il miglioramento. Siete curiosi di sapere quanto la vostra azienda sia veramente esperta di dati? Volete identificare le aree di miglioramento e valutare il livello di Data Maturity della vostra organizzazione? Se \u00e8 cos\u00ec, ho lo strumento che fa per voi.<\/p>\n

    Presentazione del Data Maturity Quiz<\/a><\/strong>:<\/p>\n