> That's the whole point of E2EE.
No it's not. The point of E2EE is that only the client apps decrypt/encrypt content, and the server just processes the encrypted bits. Most E2EE service providers do this by encrypting your encryption key. That's how you can login on other devices without you having to store and import an encryption key every time. When you login they send you an encrypted blob that contains your encryption key, which is decrypted client side, then the key is used to decrypt your data locally on the client. This does not break E2EE, but it does mean you have to trust the provider, which is why most of them are entirely open source.
Sharing and emergency access also use similar public key cryptography techniques to provide shared access to E2EE data. A similar principle applies to your phones encryption aswell, and is the reason you can wipe/reset your device in seconds, instead of minutes/hours. They only wipe the encryption key; not the encrypted data.
I think you are slightly confused about E2EE, let me try to address a few points:
- The whole point of end-to-end encryption is that nobody in the middle (e.g. the server) can possibly have access to the encrypted data. If they can (e.g. by giving you a way to reset your password), then it is not end-to-end encryption, period.
- It is possible for a provider to "encrypt an encryption key" with another key they cannot access, e.g. a password. In that case your password (probably through a KDF) is the key to decrypt the encryption key that is used further.
- It is possible for an app to store your key locally. Ideally you would store your key in a secure element on your phone for instance, and unlock it e.g. with biometry.
- In any case, if the client (e.g. the app) is not open source, you cannot easily know if it does what it says. That's why it's easier to trust Signal than WhatsApp.
- If you lose your "key" (whatever it is that the server doesn't know), your data is lost. If you can lose your key and the server can help you recover, it means that the data was not end-to-end encrypted and the server could access it without needing you at all (obviously, because you lost the part that they needed).
Now, in order to seriously trust E2EE, you need to trust the implementation. That means that if you cannot audit the code (typically, WhatsApp), then it's difficult to trust it. Another example is ProtonMail: you open the webpage in your browser, and at this point it downloads a client that is supposed to do the decryption locally. How do you trust that client? You could try to audit it, but next time you reload the page, it may download modified sources, and you won't know (there aren't tools to pin the version of a webapp from the browser).
In that sense it's even harder to trust ProtonMail than WhatsApp: it would be trivial for Proton to serve a different client that would exfiltrate your password just for you, just this one time, and no audit by anyone else in the world would ever detect it. So when you use ProtonMail in the browser, you trust the Proton server, which defeats the purpose of E2EE. It is still better than non-E2EE like GMail (where you know Google reads all emails of everybody), but you still trust the Proton server, which is not what is usually expected with E2EE.
In a recovery scenario, you don't have the password or whatever you used to encrypt the key.
And again: the whole point E2EE is that you don't have to trust the provider. Open source in general doesn't help you here because you don't know what software or which version of the code the provider is running. Your only chance is to run open source client software, review it to ensure E2EE has been implement properly, then you don't need to trust the provider.
Either the provider knows the secret, then they can help you in a recovery scenario, but then they also can read your data. Or they don't know the secret, in which case they cannot help you in a recovery scenario.