> If I got a copy of the raw recording I might consider it. Maybe. Having that audio recording would be valuable to me.
We wouldn't be able to provide it because it's never kept. It's transcribed directly, and then only the note summary is kept. This is to ensure the recording and transcript can't leak (because they don't exist). This was one of my first questions for all of these tools. Where does the data go, how is it processed, what happens. One company refused to talk about it, so I refused to talk to them.
So how can you verify correctness of transcription and summary in a way that is repeatable over time?
Agreed. That sounds like a recipe for "we don't know how 'the algorithm' came up with what it did" kinds of excuses when, inevitably, inaccuracies are found. It also seems, conveniently, to make the processing system practically unimpeachable.
Any thoughts on https://news.ycombinator.com/item?id=47896064 ?
You're not the first person to focus on the transcript, but you're forgetting that the person checking the note, the doctor, was also in the session and remembers what happens. This isn't an issue.
That's the job of the provider. There's no other way to actually verify the accuracy of the note. You can't actually engineer humans out of the loop, the loop revolves around humans.
How does the provider verify the accuracy if they don't have the transcript or the original recording?
They read the note. They were in the session, if they can't remember what happened minutes before then we have bigger problems than a lack of transcript.
You said you evaluate the error rate every month. How can you do that if you don’t have the recording or transcript?