"Setting up LLMs to output structured data is incredibly hard." resonated strongly with my experience working in similar one-off projects. I've almost always implemented some level of fuzzy-matching to validate and convert the LLM output back into my expected structured format.

I've also noticed that the LLMs are much better at writing code than structured JSON (no real surprise given the popularity of code assistants). If it makes sense in the specific situation, I now have the LLM generate code and parse it into the right structure rather than requesting structured data directly:

`generate_event("I need to do X", new Date("1-1-2025"))` seems to be more reliable to generate than `{ "description": "I need to do X", "when": "1-1-2025" }`

Hey! It's awesome to read other people's solutions to this.

I've been working on solving this for the past 2 years or so and I went through much of the same struggles in the beginning until we came up with a solution which is fairly complex, to get LLM's to output data in a way we can use.

The big problem is that 95% accuracy is not good enough for calendars. People lose confidence after 1 failed attempt. Trying to get LLM's to output JSON can have a 1 in 1000 invalid JSON problem which is unrecoverable. What I wound up doing is training models for the tasks with tremendous amounts of data. I did not use OpenAI's models as they were not right for the job. Would love feedback.

convoke.ai

I'm looking at the AppScript site.

Is this like PowerAutomate, but for G Suite products?

Also I don't know why, looking at the site makes me think it's a candidate that Google is going to kill off without warning: https://www.google.com/script/start/

Just use CALDAV; it's designed for making calendar entries automatically via email. I'm not hip with the fashion for putting an LLM into everything. I think it's lazy.

I am working on GabrielAI, which is a tool to filter and autodraft reply for Gmail and outlook and of course it uses LLM under the hood.

https://getgabrielai.com

In cases like this the author could just set up a filter like:

"If the email contains a task for me" (or some variation)

Then add a Gmail label to it.

In this way the author will immediately find all the actionable emails for him in a specific folder, much faster to skim and to keep track of all of them.

Another option would it be to have GabrielAI generate a Draft like "reply acknowledging the task and put a to-do date in the email in 1 week"

This would allow Google to track the email and the deadline.

I used to use calendar integrations for these kind of things. Then I realized I'd prefer to have the low priority stuff disappear until I have to deal with, so I switched to followupthen.com and have been happy with it. A nice effect is that it creates a paper trail so I know how many times I've put things off.

You can also do this with https://shortwave.com (and get iOS/Android/Web)

https://twitter.com/Shortwave/status/1760723475923390598

My friend and I built a gmail add on to easily parse tasks from an email and add them to your calendar.

The fact that GPT could reliably produce the right JSON structure but an open model couldn't is fascinating to me. It's impressive how far ahead OpenAI is.

I have a probably janky habit of creating slack reminders and then rolling them over by some mix of 3 hours, next day, or next Monday.

I simplify my prioritization strategies to use only 2 priorities: High (do it now) and Low (do it later). There should only be 1 high-priority item at any time, and when it's completed, the (next) low-priority item becomes the single high-priority item.

Maintaining multi-level priorities requires more decisions to evaluate relative priorities of different tasks and possible priority re-evaluation when new tasks arrive. Throw some colleagues, friends or others into the mix and agreements on the decisions become more distant.

Within the low-priority list these are sorted on the date and time required. If you then choose to ignore the priorities or sorting, the deviation will take you down the priority re-evaluation rabbit hole again. It's then your choice to follow the process, or not. Avoiding adding complexity to task scheduling and processes ensures I have focus.

Of course, this will not be for everyone. Good luck with that LLM!

structured output by LLM can be achieved by using the Python outlines library

[dead]