TransWikia.com

Applying machine learning to cache algorithm

Data Science Asked by Lem0n on November 9, 2020

I have an application that receives ~10k of requests per day; each request has multiple parameters and goes through a pipeline with multiple steps and they finish in about 1 hour.
Let’s say the requests are like bake_cake [flavor=chocolate] [topping1=strawberry] [topping2=cream]

Given a history of requests from previous days, is there a ML framework that can help me predict the “100 most like requests to arrive today” so I can cache their results? Or some other similar strategy that can help me delivering part of those requests faster?

Ideally it would be based on how much I gain by delivering faster, how much I lose by processing a request that might not come, etc.; but for now even simpler algorithms could be of much help since currently there’s no optimization.

One Answer

You need to validate your assumptions based on data driven analysis

  • Types of requests received on a day
  • Types of requests received on a holiday
  • Pattens for different times (Morning / Evening / Night)
  • Seasonality / Monthly / Weather based patterns

Post Analysis

  • Do you see correlation from Data & patterns
  • Do you see correlation from Data & Location
  • Do you see correlation from Data & Age / Gender / Locality

ML Models

  • Do you see correlation from Data & patterns
  • With the variables we can identity and model a forecast algo

ML Model is not the start. It is the outcome of detailed data and domain driven analysis

Answered by Siva on November 9, 2020

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP