Ask HN: How much would it cost to replicate GPT-2's model?
Cost 1: Scrape 8 Million articles from Reddit posts.
Cost 2: Training a 1.5 B parameter transformer network.
I'm assuming it could be trained with google's TPUs. And the model would just be a bigger version of the model open ai released.