Projects

Ops

• nix - A cross-platform setup script that works with both Linux and Mac.
• self-hosted - Self-hosting open-source alternatives for popular services. Managed via docker-compose.
• terraform-sops-ssm - Create SSM secrets from SOPS-encrypted secrets, with IAM roles & users creation for SSM access.
• Vercel - Multi Branch Deployment - Use GitHub Actions to deploy a frontend project from different branches (dev, uat, master), each with their own preview environment.
• pgconn - pgcli wrapper to connect to PostgreSQL database specified in db.yaml. Proxy/tunnel connection is automatically created and killed when pgcli is exited.
• GKE Autopilot Cost Calculator - Calculate GKE Autopilot workloads cost. Available for normal application deployments and spark-submit jobs. Source code here.

Data Engineering

• Dataframe Frameworks Showdown - Benchmark performance between duckdb, polars and spark. In addition to runtime, RAM usage is also provided.
• Spark on Kubernetes

Data Science

• Impute Pipelines - Use machine learning to fill in missing data. Utilize hyperparameter tuning to find the optimum parameters.
• Visualizing Map Region Prefix/Suffix - Utilize NLP to group region name prefix/suffix.
• Word-Based Analysis With Song Lyrics - Visualize lyrics trend using NLP and use topic modeling to find common words per specified clusters.

Tools

• music-lyrics-tagger - Add lyrics to flac and m4a files.
• subsonic-github-readme - Now playing and random tracks widget via subsonic API. Golang port here.
• todotxt-to-calendar - Convert todo.txt entries to calendar all-day event.
• water-cut-notify - Send water cut alert as LINE notifications.