Show HN: Truss – Serve any ML model without boilerplate code
68 points by philipkiely 2 years ago | 9 comments- 0xferruccio 2 years agoThis looks promising. It feels like for non ML engineers it’s very hard to figure out how to use models as part of vanilla CRUD codebase.
For instance in a Rails app the ML model services would probably be served as a completely external service API generated with something like Truss wrapped in a service class that just exposes the outputs and handles errors/input validation!
- ricklamers 2 years agoIn this category I’m a big fan of https://github.com/bentoml/BentoML
What I like about it is their idiomatic developer experience. It reminds me of other Pythonic frameworks like Flask and Django in a good way.
I have no affiliation with them whatsoever, just an admirer.
- philth 2 years agoBig fan of projects like BentoML, the whole ML space is powered by some truly great projects. What we've tried to do with truss is create an experience that is optimizing for simplicity and universality. Our roadmap begins to deviate away from projects like BentoML, and so creating truss was very much motivated by the need to continue this mindset as we march down through that roadmap.
context: I am one of the contributors
- philth 2 years ago
- d136o 2 years agoSuperb product and team.
Worth looking into if you’ve done any engineering work around deploying ML models as a or within a service.
- gorg93 2 years agoLooks interesting, what if I need to write some logic (pre/post prediction) in the prediction server?
- b10_bola 2 years agoEvery time that you make a truss to package your model, you can add custom pre/post processing to every request/response. We've done our best to document it. https://truss.baseten.co/reference/structure#modelhttps://truss.baseten.co/develop/processing
Take a look at `{truss_name}/model/model.py` to add the code.
Please open up an issue or PR if you think we can improve on the documentation!
- b10_bola 2 years ago
- dweinus 2 years agoLooks great! What is the argument to use this over MLFlow model packaging and serving?
- philipkiely 2 years agoTruss contributor here
MLFlow is awesome; it's powerful and flexible and extends through multiple valuable portions of the ML cycle.
Truss aims to be simpler to use in both straightforward and complex cases, requiring fewer lines of code but more importantly coming in later in the modeling process. Truss is opinionated and focused.
But interoperability is important to us and we are looking into ways to connect Truss and MLFlow.
- philipkiely 2 years ago
- jphoward 2 years agoThis is likely to share its name with the next Prime Minister of the UK...