Data Platform for Metaverse and Social Game Company

IMVU Inc. is an online metaverse and social game. IMVU members use 3D avatars to meet new people, chat, create, and play games. IMVU hosts 6M+ active players, and a virtual goods catalog of more than 40 million items, the largest catalog of virtual items in any game platform. The number of concurrent users reached 150k in April 2020. 

A Proxet team has been working on architectural and engineering improvements to IMVU’s data platform, as well as migrating IMVU’s data center from its own bare-metal servers onto Amazon Web Services.

What we did:

1. Implemented a scalable data platform to accommodate continuous growth in data volume.
2. Re-architected data ingestion to minimize data loss and achieve reliable delivery for over 99.999% of data.
3. Improved data quality, and implemented advanced checks and validation.
4. Implemented data analysis architecture that is scalable, available, consistent, fault-tolerant, and cost-effective.

Technology

• Cluster orchestration: Kubernetes (Amazon EKS)
• Data formats: JSON (raw data), ORC / Parquet (in the data lake)
• Monitoring / alerting: 
  • Prometheus (time-series database / alerting)
  • Grafana (visualization)
  • Elastic stack (log ingestion / processing / visualization)
• Data lake: Amazon S3 / EMRFS
• Message queue: Apache Kafka
• Job orchestration/scheduling: Apache Airflow
• ETL: Apache Spark / Hive (Apache EMR)
• SQL engine: Presto
• BI: Tableau