Cost Of Self Hosting Llama-3 8B-Instruct

Self-hosting an LLM with EKS costs approximately $17 per 1M tokens versus ChatGPT at $1 per 1M tokens. Using four Nvidia Tesla T4 GPUs can reduce the cost to <$0.01 per 1M tokens but takes ~5.5 years to break even. Factors like hardware management and usage frequency must be considered.