Claim in Origins of modern language models
GPT-3 has 175 billion parameters.
Verdict: supported
Evidence for
“Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting.”
Note: Abstract of the original GPT-3 paper (Brown et al., 2020), arXiv:2005.14165.
“we train GPT-3, an autoregressive language model with 175 billion parameters”
“GPT-3 has 175 billion parameters, each with 16-bit precision, requiring 350GB of storage since each parameter occupies 2 bytes.”
Note: Wikipedia GPT-3, paragraph 24; second primary domain beside the two accepted arxiv.org items (same abstract). The NeurIPS 2020 proceedings abstract (snp_3u5039274a3b0f, paragraph 5) also says 175 billion, but it is the same authors text, so I chose an independent source.
Evidence against
None yet.