Detangle: AI-generated summaries of legal docs
112 points by Shpigford 2 years ago | 100 comments- fellerts 2 years agoWho is the target audience for something like this? I'm not sure anybody is going to pay to have Apple's ToS simplified before clicking "agree". On the other hand, if you are squinting at a contract that actually matters, I don't think a service like this gives sufficient understanding of what all that legalese would actually mean if shit hits the fan. For example, changing the applicable law in a contract can yield wildly different interpretations without changing a word in the contract iself. This is why you should seek professional help when in doubt.
You say "The summaries are to help inform conversations with actual lawyers", but would a lawyer give a rats ass about a layman's interpretation of an AI-backed translation of a legal doc from a company who "does not guarantee that the stuff on the website is good or works"? IANAL but I don't think so.
- NoboruWataya 2 years ago> You say "The summaries are to help inform conversations with actual lawyers", but would a lawyer give a rats ass about a layman's interpretation of an AI-backed translation of a legal doc from a company who "does not guarantee that the stuff on the website is good or works"? IANAL but I don't think so.
If you are conversing with your own lawyer, you should be getting your summary from them, as that way you can be fairly sure it's correct (and you can probably sue them if it's not). That's part of a lawyer's job, explaining the law to their clients.
If you are conversing with someone else's lawyer, and you don't have your own lawyer, you are dangerously close to relying on this software for legal advice, which, given how confidently incorrect AI can be, could end in tears.
- fnordpiglet 2 years agoI assume lawyers could benefit from a tool that can summarize a large document rapidly. They’re often faced with thousands of pages of documents, and in large class actions possibly hundreds of thousands to millions.
- traceroute66 2 years ago> I assume lawyers could benefit from a tool that can summarize a large document rapidly. They’re often faced with thousands of pages of documents, and in large class actions possibly hundreds of thousands to millions.
That tool is a human called a Junior Associate.
And, no, you cannot replace a Junior Associate with AI for the reasons already outlined by @kkielhofner below. TL;DR: context... Junior has it. AI doesn't and will never do.
- alexdoesstuff 2 years agoI imagine for use cases such as class action there may be a place for AI to effectively extract information from hundreds or thousands of similar but slightly different contracts. Same goes for other such situations, such as commercial landlords with significant rent portfolios, investors/banks with loan terms, or even investors more easily understanding the difference between various bond and/or loan terms.
One of the key advantages of the recent advances in AI/ML to me is that it allows users to more efficiently extract unstructured information.
- 2 years ago
- traceroute66 2 years ago
- bronco21016 2 years agoI’m a unionized employee who is currently waiting on finalized legal language for a new contract.
A lot of my time will be spent reviewing a diff between the current agreement and the proposed agreement but it’s a 500 page document. I might give this product a try just to see what it spits out. I could see this being incredibly useful if it could write a plain language, non-biased, summary of changes.
- muttantt 2 years agoA 4 page document prompted me for a $50 payment.
Good luck running 500 pages through this thing...
- kkielhofner 2 years agoThe pricing is absolutely disconnected-from-reality wild.
At worst a junior associate at $BIGFIRM would review a document of that length with an understanding of issues specific to your situation, governing law/locality, etc for somewhere around $100 AND this would include a redlined response with suggested edits, annotations, any questions, etc.
A local, friendlier more SMB or startup catering attorney/smaller firm would likely look at a bunch of these for free or close to it.
In short the pricing needs to be at least 1/10 what it is currently so it can (at best) serve as a novelty, a basic filter to a series of potential agreements, etc.
- kkielhofner 2 years ago
- mc32 2 years agoI would see if it could interpret Oxford commas appropriately such as in: https://www.nytimes.com/2018/02/09/us/oxford-comma-maine.htm...
- muttantt 2 years ago
- traceroute66 2 years ago> You say "The summaries are to help inform conversations with actual lawyers", but would a lawyer give a rats ass about a layman's interpretation of an AI-backed translation of a legal doc
Indeed.
We all know what the medical profession quite rightly thinks about patients who turn up having (supposedly) self-diagnosed themselves with the help of Doctor Google.
- arcticfox 2 years ago> We all know what the medical profession quite rightly thinks about patients who turn up having (supposedly) self-diagnosed themselves with the help of Doctor Google.
And we all know what artists think about AI art...they hate it, because it's pretty good and it's going to take a lot of their jobs.
"Doctor Google" is pretty bad but there's absolutely no reason AIs can't get very good at most areas of medicine. For now doctors don't like Google because it gives bad results, but pretty soon they're going to start hating it because it gives good results.
- arcticfox 2 years ago
- NoboruWataya 2 years ago
- falcolas 2 years agoGiven how confident AI can be when it's incorrect, it feels like a bit of a problem to use AI with legal docs, since you can't really trust it to be correct (and the cost of being incorrect may be absurdly high).
- drbwaa 2 years agoGiven how _poorly written_ a lot of real legal documents are, I would definitely not trust any current AI to summarize it accurately. How would it handle documents where terms or clauses are in conflict, or any other situation where a close-reading human would have to step back and say "okay, this is _semantically_ impossible?"
I could see a tool like this being valuable as an outline generator for long documents, but I would be very reluctant to believe any statements it makes about the actual legal effects.
- CSMastermind 2 years agoI could see this being useful as a tool that scans legal documents in bulk like Terms of Service and looks for red flags that simple keyword searches would miss.
- intuxikated 2 years agoCould definitely be a useful tool to improve something like ToSDR
- intuxikated 2 years ago
- CSMastermind 2 years ago
- scrose 2 years agoI’ve had ChatGPT tell me how to do something one way, and then give an example that omitted the specific syntax it had called out. It was only until I told it that the example didn’t reflect what it wrote that it basically said ‘oops! Here’s the corrected version’. I can’t even begin to imagine how poorly that could go in a legal context.
- tluyben2 2 years agoWith chatgpt, with the little toy project I am trying that uses it, I default prompt it (automatically) that is is wrong and please provide a better version. I will generate a few versions that get better. Mind you that I don’t tell it what was wrong saying ‘this does not include all the points I asked’. It will say ‘oops, here is a version that includes all you asked’. And that a 3-4 times in a row.
- tluyben2 2 years ago
- make3 2 years agoAs a NLP researcher at a well known place & working on this, I totally 100% agree with you, this is ultra dangerous and should not be used, straight up, on anything remotely serious
- babyshake 2 years agoThis seems targeted at the low end of the market, where the alternative is simply to not read a ToS, EULA, etc.
- NoboruWataya 2 years agoI highly doubt the low end of the market will pay anything close to what this service wants to charge.
- fnordian_slip 2 years agoWouldn't no information be preferable to wrong information in most cases?
- NoboruWataya 2 years ago
- oa335 2 years agoAgreed, I would still want a lawyer to take a look or anything of importance.
- Shpigford 2 years agoI also would want a lawyer to take a look. Hence the note saying to talk to a lawyer. ;)
The reality is, even knowing what to ask a lawyer can be difficult and it's not economically viable to ask a lawyer to summarize a 100-page legal document into text you can wrap your head around.
Detangle just helps give some clarity so that you can further a conversation with a lawyer, or even just ask for clarification from the person who sent you the document.
"Hey, this paragraph seems to imply XYZ...is that the case?"
I'm making zero claims that this should replace a lawyer or be used as a replacement for legally binding text.
- traceroute66 2 years ago> even knowing what to ask a lawyer can be difficult
That's the lawyer's job, to guide you through ! If they're not doing that, then find a different lawyer.
The whole point is we are talking about legal document review here.
Reviewing legal documents necessarily involves an active discussion with a lawyer, it is not something you can turn up with pre-prepared questions for.
- notahacker 2 years ago> The reality is, even knowing what to ask a lawyer can be difficult and it's not economically viable to ask a lawyer to summarize a 100-page legal document into text you can wrap your head around.
A 100 page legal document (i.e. one with lots of stuff implied by structure and lots of specific subconditions and lots of case-specific stuff that might make more sense in conjunction with the examples in the appendix) sounds like the sort of thing AI would be wrong about far more often than it was right...
At least reading a short contract or EULA I'd be pretty confident an AI-based system would have lots of similar contracts and plain English explanations in its corpus and correctly identify the boilerplate exclusions and even if it misinterpreted stuff like IP assignments and ability to modify the agreement it might be slightly better than Ctrl-F at figuring out which clauses do this. Although from what I can see the current incarnation is working on a paragraph by paragraph basis so doesn't even necessarily apply definitions defined in the section above correctly...
- jchw 2 years agoQuestion: how much would it cost to consult a lawyer about a 100 page document? How does one even go about doing something like this? I've never really needed to do that, so I'm genuinely curious.
- traceroute66 2 years ago
- Shpigford 2 years ago
- stronglikedan 2 years agoThat's why all the results come with an "AINAL" disclaimer.
- falcolas 2 years agoAah, yes. The old fine print switcheroo.
"Here's a tool for this highly specific purpose!†"
"†Tool not fit for this highly specific purpose."
- falcolas 2 years ago
- drbwaa 2 years ago
- tasseff 2 years agoCan this idea be turned on its head to engineer adversarial legal language that sounds reasonable but actually has terrible implications for the signee?
- lordgrenville 2 years agoI've heard that in situations like this courts will usually uphold the spirit and not the letter of the contract (same as eg if you have a typo and add a zero somewhere). Not a lawyer and might be 100% wrong on this.
- awb 2 years agoINAL, but I think this is true to a certain extent.
I’m pretty sure for a contract to be legal both sides need to receive “valuable consideration” (something of value). So, if you get completely screwed over, I think there can be a legal remedy.
But, I don’t think there’s much a court will do if you come out on the losing end of a contract, just based on the terms themselves as long as both parties still received something of value.
- singleshot_ 2 years agoI’d say it has more to do with the notion that a contract that represents a unilateral mistake of which the no mistaken party knew or should have known of, then the contract is voidable by the mistaken party.
After all, it’s unlikely that you’re going to get no consideration in a contractual transaction… but if you get absolutely nothing, sure, the contract could be void for lack of consideration.
- singleshot_ 2 years ago
- zopa 2 years agoThere’s a rule of thumb (“canon of construction”) that ambiguities in an agreement are construed against the drafter.
- awb 2 years ago
- 2 years ago
- make3 2 years agoI'm sure it would work with a LLM if you give it a few examples
- lordgrenville 2 years ago
- traceroute66 2 years agoI'm sorry, but in the nicest possible way, this is bullshit.
Anybody who has been in business-roles that involve reviewing legal documents will tell you that.
TL;DR three things:
Really, I'd run away from something like this as fast as you can.1. The AI will NOT tell you if it is the right document for the context, only a lawyer will do that 2. The AI is unlikely to correctly analyse the document, clauses in a legal document have inter-relation with each other and with the general context (perhaps even with other legal documents you have previously signed). That sort of analysis is only something a lawyer can do. 3. Perhaps MOST IMPORTANTLY ... with legal documents, often the important thing is not what is IN the document, but what is NOT IN there. Only a lawyer can tell you what is missing in a legal document AND help you get that missing stuff negotiated into it.- kemitchell 2 years agoWhether you're using the right start point and picking the right optional pieces are definitely important. But for most kinds of agreements I see and advise on, there's nothing lawyers know that nobody else could know. What you need doesn't have to come from a lawyer, though it often does.
Meanwhile, specifics of the deal and the broader context can also matter, sometime more than the abstractions legal forms tend to deal in. A lawyer may or may not notice those and reason through them.
Paving an AI path over ill-fitting or over-standardized legal terms isn't great for the industry. Neither is reinforcing lawyer monopoly over business knowledge.
- jackbrookes 2 years agoWhy do you think AI cannot be as good as a lawyer for all these things? Take the technology to its logical end and it can be better than a lawyer
- notahacker 2 years ago> Why do you think AI cannot be as good as a lawyer for all these things? Take the technology to its logical end and it can be better than a lawyer
Because the "logical end" of an LLM that can match words with other words isn't even close to AGI that actually understands the current version of the relevant law and its specific implications for a specific entity in a specific jurisdiction. But it can spout convincing looking bullshit based on having parsed a load of text with a certain amount of semantic similarity which is completely irrelevant in legal terms.
A random teenager on Reddit who prefaces his posts with "IANAL, but I think..." is closer to the relevant level of understanding of legal documents and jurisdictions than GPT-3, but nobody ever wonders why people don't think teenagers on Reddit are better at offering legal advice than lawyers...
- notahacker 2 years ago
- kemitchell 2 years ago
- ahepp 2 years agoHow is this "not legal advice"? You're _selling_ summaries of legal documents.
- superfrank 2 years agoI assume their argument would be that they are just summarizing information that you provided and not giving you advice on what to do with this information. Similar to how summarizing a book's plot isn't an endorsement of that book.
I'm not a lawyer though, so I have no idea if that logic would hold up in court. I'm just guessing that's the rational they are using.
- superfrank 2 years ago
- denton-scratch 2 years agoI started wading through the Google example. I got about 0.25 of the way through.
The glosses all seem accurate, if incomplete sometimes. I got a strong sense that they had been reviewed/polished by a human "expert". That might explain the pricing.
I didn't find the glosses much clearer than the original, just shorter. The Google legalese I read was fairly clear. With the document being a legal contract, it's obviously vital to understand the legalese.
I'd be impressed by an AI that can read an employment contract, and say "Whoah! Clause 31-B(ii) is implicitly asking you to sign your descendants into servitude in perpetuity! Perhaps you should take a look!", or "Did you notice that all benefits are contingent on stuff beyond your control?". I'd probably have paid $25 for that, because I've never had a lawyer scan my employment contract. I read them myself, but IANAL.
- goldforever 2 years ago[dead]
- goldforever 2 years ago
- tluyben2 2 years agoYou would actually want to discuss the document with a lawyer (AI), for instance asking if not paying the 100k in point 3.4.7a is enough for termination of the contract, show the reasons (copied from the doc and explained) and if yes, are there any clauses that invalidate that clause or any logical inconsistencies regarding that clause etc.
As someone (no legal background) who has to read legal documents sometimes (perks of being a founder cough) and would take advice (yet) from an AI over a lawyer; the above would give me enough of an idea how exposed we are or not and if a lawyer should be contacted. I can read legalise quite well (perks of having been a founder for 35 years), but if its 100s of pages, I need to be able to ask questions about the doc to get an idea of where I stand. Which is why I would hire a lawyer to do a review after I send them my questions anyway.
- peter_d_sherman 2 years agoI uploaded a copy of the U.S. Constitution...
It told me that it would cost $123 USD to summarize...
It also told me:
>"This document summary is not legal advice or legally binding in any way. You should consult a lawyer."
Opinion:
A Lawyer would cost me less... <g>
(Perhaps I could get a cheaper rate if I uploaded the Magna Carta or The Twelve Tables (of Ancient Rome) or the Ten Commandments or Newton's Three Laws -- or the Two Laws of Richard J Maybury:
o Do all you have agreed to do
o Do not encroach on other persons or their property
)
All I know is, $123 is too expensive for me -- and,
What if the AI takes the Law -- and interprets it all wrong?
?
???
- bernardv 2 years agoThe fact I have to upload to a server and my document will not be deleted for up to 24 hours, makes it a no go for most corporate users. Make this a standalone desktop app, with zero cloud dependency and it would be useful. This is very useful for a first pass through a legal doc, for example, to triage documents.
- binarymax 2 years agoAlso, it uses OpenAI services (GPT3) - so now you also need to understand their TOS and data retention policies. Makes it a non starter for any private or confidential documents.
- binarymax 2 years ago
- nashashmi 2 years agoI work through a bunch of contract documents reissued with a just a few changes. I need a service that would identify what the changes are instead of me going through each document and reading it over again Or doing a comparison of previously issued documents with this one.
- NoboruWataya 2 years agoWhat do you mean exactly, are you just looking for software that will highlight the differences between two documents? That software exists already.
- nashashmi 2 years agoYes that software does exist and currently my go to method (convert pdf to word document, compare changes in word).
The ideal system would highlight (sets of) text that that's common between both documents and underline text that is original. Or vice versa.
- freejazz 2 years agoYou can run redlines of PDFs in Adobe Acrobat pro, no need to conver to word.
"The ideal system would highlight (sets of) text that that's common between both documents and underline text that is original. Or vice versa."
Just change your settings in microsoft word.
- freejazz 2 years ago
- nashashmi 2 years ago
- yakshaving 2 years agoInteresting, what is your job? Contract lawyer? do you work in biglaw or in house counsel? or a vc firm?
- nashashmi 2 years agoEngineering marketing concentrating on government proposal requests. That involves contracts, described scopes of work, etc.
- nashashmi 2 years ago
- meindnoch 2 years agogit diff --word-diff
- NoboruWataya 2 years ago
- Shpigford 2 years agoDetangle gives you AI-generated summaries of your legal docs so you can understand what you're signing.
Here's the YC SAFE, for example: https://detangle.ai/examples/yc-safe
- simonista 2 years agoI'm excited to see where this goes because I think it is a cool application of AI. That said, two of the first three paragraph summaries are wrong in this example:
> [Investor Name] gave [Company Name] the right to certain shares of its Capital Stock in exchange for [Amount] on [Date].
This is backwards
> The Post-Money Valuation Cap is a number that is written in Section 2.
This isn't true, the number is here, "additional defined terms" are later.
- treis 2 years agoIt's anywhere from slightly to very wrong for each section. Pretty cool as a tech demo but definitely should not be used to understand a legal document.
- treis 2 years ago
- andrewfromx 2 years agoI wonder, what happens when you read just the summary of a complex section, sign, and then years later are in a lawsuit. If Detangled's algorithm in shown to have clearly written the summary wrong... are they in any way liable? hehe notice their own terms with summaries: https://detangle.ai/terms
- NoboruWataya 2 years agoFrom the original text:
> We may suspend or terminate your right to use our website and terminate these Terms of Service immediately upon written notice to you for any breach of these Terms of Service.
From the summary:
> We can end this agreement any time we want.
I read these as being quite different? The original text says they can only terminate for a breach by the user of the ToS, not simply "any time we want".
I get that the service isn't supposed to replace actual legal advice but surely differences as glaring as these limit the usefulness of the summaries.
- Shpigford 2 years agoThat was the big reason for adding the massive disclaimer everywhere, but I guess we'll see.
I also intentionally didn't just give an output of the summary but instead showed it next to each paragraph so the implication isn't that the summaries replaces the legal, but rather tries to clarify.
Glad you noticed the Terms. :)
- NoboruWataya 2 years ago
- ahepp 2 years ago> AI-generated summaries of your legal docs so you can understand what you're signing.
If you sell me a summary of a legal document, which was advertised as being useful to understand the legal document, that seems like just about the most straightforward case of legal advice I can imagine.
- ryanlitalien 2 years agoAbsolutely fantastic! This is a great use of AI here, definitely using this going forward. A bit concerned though, the LLC name doesn't give confidence when reading through the output with confidence.
- simonista 2 years ago
- alexmolas 2 years agoWith these prices they could have a lot of people in some cheap country summarizing the documents.
- xiphias2 2 years agoIt wouldn't be a bad idea either
- xiphias2 2 years ago
- freejazz 2 years agoIt says all documents and summaries are deleted after 24 hours, but are the contents of the documents incorporated into a model? I'd imagine for some kinds of legal documents (say those under a court's protective order) this would present issues.
- adamsmith143 2 years agoAre you able to convert the weights of a Neural Network into the training text that adjusted them?
- freejazz 2 years agoSimply allowing someone to do this could violate the court's order, is the point. Maybe detangle should give a summary on the kind of documents it can help with and how to identify them, if it's going to pretend to be anything like a lawyer.
- adamsmith143 2 years agoI'd question if that would hold up in court. Can the court order also be violated if I burn a contract and use the ashes to grow vegetables? Presumably about as much information in the contract is extractable from the vegetable as from a NN weight.
- adamsmith143 2 years ago
- freejazz 2 years ago
- adamsmith143 2 years ago
- planede 2 years agoUh, I wanted to try it with GPL v3, but no way I'm paying $111 for it.
- jmacd 2 years agohttps://www.spellbook.legal is another one I have seen that is doing the opposite (helping with the writing of contracts)
- t_serpico 2 years agoanyone else worried about how tools like this will make us dumber?
- quaintdev 2 years agoIt depends what you use the tool for. You can use tool for menial tasks while doing something highly creatives. Tools inherently don't make us dumber. The choice is always lies with the individual.
- 2 years ago
- quaintdev 2 years ago
- nerdponx 2 years agoWhat are they doing with all the uploaded document data? I didn't see that in their terms of use.
- Shpigford 2 years agoWe use GPT-3 to summarize it the document data. Then 24 hours later we completely delete all text, URLs and file uploads.
- Shpigford 2 years ago
- Ryanb58 2 years agoLove it! This is one of the best uses of ML/AI IMHO. I really dislike reading leasing agreements.
- Shpigford 2 years agoThanks so much!
- Shpigford 2 years ago
- jamesdwilson 2 years agoDoes anyone know of a "Tangle" - it would be neat to do the inverse!
- mtlmtlmtlmtl 2 years agoHow is this tested to make sure the summaries aren't wrong?
- muttantt 2 years agoI tested with a 4 page basic agreement. Price quoted was $50+.
- quaintdev 2 years agoThe summaries are good but I wish it also had the ability to answer my questions about document like whether something is allowed or not or what are my rights, etc,.