At the start of July, I knew I wanted to build a new project but the idea wasn't 100% decided and neither was the tech stack. So, when I heard about the Hashnode and PlanetScale hackathon happening the timing couldn't have been any better. With the tech stack now decided, I trawled through my backlog of project ideas and thought about what would be cool to build with PlanetScale. In the end, I decided on an analytics app for GitHub.
This app would soon become a full-blown product that I built in public over on my Twitter. Eventually, the app became known as Squares. So, this is the story of Squares and the journey I went on building it.
Problem Statement & Inspiration
As a self-proclaimed data nerd and lover, anything with analytics and data about me and my behaviors is instantly an interest. So, knowing how much time I spend on GitHub and contributing to projects on it, I wanted to know more about my behaviors and activities. This is where the idea for Squares was born.
Essentially, I wanted to make Squares into a single one-stop shop for all of your GitHub analytics and data needs. Regardless if it was at the repository level or the user level you would be able to see your data and interact with it. It would also let you directly access the pages of the data it was reporting on. So, for example, you could see a commit that has 1500 additions and click on the GitHub icon and be taken directly to that commit and see the exact changes within it.
In short, a dashboard to interact with and report on all your GitHub activity.
Applications
My end goal and vision for Squares is for it to become a regular tool for reporting on progress and trends across both individuals and teams with the ability for people to make their own graphs, charts, and visuals.
Now, admittedly we aren't there yet and the current MVP of Squares is just laying the foundations for this future. So, right now as mentioned it's a single place to see all your data to browse and interact with it. But, the post-MVP vision is much more than that.
Features
Right now, we have two key features in Squares, one is the repository view and the second is the user breakdown view.
Repositories View
At the press of a button, you can see the contributors, commits, languages, and statistics for your chosen repository. You can then look through it and if anything catches your eye you can instantly jump to it on GitHub to see more about it or interact with it.
User Breakdown
On the user page, we can see the languages breakdown for the user which is taken from all of their repositories. This allows us to give an accurate view of the languages they use most and in what percentages.
Second, on this page, we can see the user's most recent repositories sorted by their push date. In the future, I'd like to expand this page to give more user-level reporting and visuals like how their commits across the last month breakdown per repository and stuff like that.
Tech Stack & Setup
The tech stack and setup of Squares was an interesting journey, so I'm going to break it down per area, frontend, backend, and general setup.
Frontend
On the frontend, Squares (both the app and marketing website), use Next.js and TailwindCSS. This combination is something I've really grown to love over the last few months and I don't think I could build a project without them now.
Next.js is almost too powerful now... And, TailwindCSS needs no introduction, it's become a staple in my arsenal of tools and lets even non-designers like me build beautiful websites in no time at all. 😍
Backend
Obviously, the database was already chosen for this hackathon, PlanetScale. But, I decided to pair it with the ORM Prisma and all I can say is wow, they make one hell of a combination. The two of them together are beyond powerful and make building a full stack application a dream and pleasure to do. 🔥
The developer experience of both Prisma and PlanetScale is truly amazing and something I've really enjoyed this month while building Squares.
The final piece of the backend is authentication which we use the NextAuth.js package coupled with its GitHub OAuth provider. So all you need to sign in to Squares with is a GitHub account then all the necessary data is stored in PlanetScale via Prisma.
General Setup
Because I don't want to stop building products at Squares, I wanted to treat this as a proper SaaS product and do almost a dry run for future projects and ideas I might want to build. So, this meant building out a marketing website to attract customers and then the actual application for people to use.
Monorepo
I didn't want to host the two sites on separate repositories because while it's a valid approach I knew there would be a lot of overlap between them. This overlap was especially on the frontend with TailwindCSS but also in other areas like my eslint and TypeScript configs. So, this lead me down the path of building a monorepo which was a new experience for me and one I have to say I really enjoyed doing.
For Squares, I decided to use Turborepo by Vercel. Overall the experience of using Turborepo was great, it made developing two custom Next.js applications alongside each other a pleasure and made running linting and test commands across both apps a joy.
All I needed to do was run the one command at the root and it handled the rest for me. Even setting up the hosting of the two separate applications on Vercel was easy once I found the right commands to use. Overall, I really enjoyed using Turborepo and would happily use it again for future projects.
Multi-Environments
Now, it was probably overkill for this project but I wanted to do things properly as if this was a real SaaS product. So, this meant not just throwing stuff into production and seeing if the application breaks or not. (Although, this was very tempting. 😅)
So, what I went and configured was your traditional multi-environment setup for projects. That is a local development setup isolated to just my machine, then a staging environment on Vercel to simulate the actual production environment, and then finally the actual production environment that customers would use.
Setting up the local development environment and production environment was a breeze with Planetscale. I had two branches in PlanetScale, the main branch set as a production branch and then another dev branch. The dev branch I used locally for development and then when I was ready I would do a deploy request to main. At the same time as doing the deploy request, I would merge the PR to main on GitHub which would trigger a production build on Vercel to update the production environment.
However, when it came to setting up the staging environment it was a bit more involved and probably should have its own post explaining it. 🤔 But, the overview is, that I use the latest preview deployment on Vercel as the staging environment and connect it to the dev branch on PlanetScale. Now, ideally, I would use a separate staging branch in PlanetScale to truly replicate production but you're only allowed one development branch on the free plan so... 🤷♀️
That sounds pretty simple so far so where is the complexity you might ask? Well, the complexity comes with using GitHub as the OAuth provider. On GitHub, I have three OAuth apps, one for local development, one for staging, and one for production. For the OAuth to function correctly you have to point it to a callback URL which for local development and production isn't an issue, I can use localhost:3000 and app.squares.so for them respectively.
But, for staging, I'm using the latest preview deployment on Vercel which all have their own random URLs generated on deployment. So, how do you set up the callback URL to work with that? Using domain aliases and the Vercel API, that's how.
I wrote a custom bash and JS script that runs on GitHub actions for each PR open/update and pushes to main. This script triggers the build on Vercel and then once complete if it's not been triggered by the main branch, it assigns a domain alias of staging.squares.so to the deployment. This means the latest preview deployment on Vercel will always have the domain staging.squares.so pointing to it. So, in GitHub, I can use this domain and the OAuth will function correctly and independently from the local development and production environments.
If you're interested in seeing the scripts for this, here is the link to them on GitHub.
Testing, CI, and more
Testing is a major part of any application and I have set up Jest tests to work on both the Squares app and website. I have written a couple of basic ones for the marketing website but unfortunately, I didn't get the time to write more for the marketing website and the main application itself but this is something I will 100% be returning to do.
Also on my list for testing are Cypress E2E tests and integration tests with Jest, both of which will use a Docker MySQL container to simulate the PlanetScale database for testing purposes.
For the CI, I use GitHub Actions. I currently have two actions set up, one for triggering the Vercel builds mentioned above and the other for running eslint, commitlint, Jest, and building both applications.
I also want to expand out the CI, to include semantic-release in the future so I can have tags, releases, and changelogs automatically generated for the monorepo as new features and bug fixes are added to it.
Finally, I have a few husky hooks setup in the repo, these are pre-commit for running eslint and prettier checks, pre-push for running Jest tests prior to pushing, and commit-msg for linting the commit message.
Challenges & Lessons Learned
Throughout the building of Squares, I encountered various issues and challenges that taught me a great deal. In this section, I want to outline a few of those and the solutions I came up with, and/or the lessons I learned from them.
Vercel domain aliases and environment setup
I won't go over everything about the setup of the environments and aliases again here but setting this up was a good challenge and not something I thought I'd end up doing when I set out on the project. I mean I knew I wanted the multi-environment setup but I didn't think I would end up writing bash scripts and learning the Vercel API to do it.
In the past I've relied heavily on Vercel's great integration with GitHub to do it all for me but having to do it manually via their API and GitHub Actions was refreshing and a great learning experience. It highlights just how powerful their API is and some of the awesome things you can do with it.
Configuring a monorepo
Again, I won't go into too much detail here as I covered it earlier in this post but prior to this hackathon, I had worked in but never actually configured and made a monorepo from scratch. It took me a few days to get it all set up and working but it was 100% worth it and something I'm glad I did and would do again for future projects.
Vercel timeout issues
This issue is actually a very recent one, when deploying the application to production in the last few days of the hackathon I ran into Vercel's time limit of 10 seconds for serverless functions on Hobby plans. This was a pain and the solution for me right now was to upgrade to the Team plan and get an increased time limit of 60 seconds on my serverless functions.
Other than how to spend money, this issue did teach me a lesson. And, that is Next.js API routes aren't the best solution to all problems. Which in hindsight makes a lot of sense... 😅
Right now, in the application, everything from sourcing data from GitHub to writing it into PlanetScale and then retrieving it is done via API routes. Now, the retrieving and writing isn't the issue really, that happens fairly fast and doesn't run into the timeout issue.
But, what does run into that issue is fetching data from GitHub, transforming it into the data structures I need and processing it. For example, for commits I need to get all of them in a repository and then iterate over each commit, looking it up on GitHub's API to get the stats and further info about it before writing it to PlanetScale. So if you scale this up to hundreds of commits, it can easily take some time to complete.
So, given the opportunity to redo it (which I'm planning to do post-hackathon), I would move the data sourcing from GitHub and writing into PlanetScale onto something like AWS Lambdas, running on a daily schedule. This means we don't have a 10-second timeout limit and don't impact the user loading times making them wait. Then the Next.js API routes would load in the already written data from PlanetScale via Prisma.
Efficient Backend Planning
Following on from the last point, going into this project I was under no illusion that I am very much an entry level developer for backend work. Sure, I can spin up an API and write data to and from a database. But, this project really showed me I have a way to go before I'm proficient at designing efficient and scalable APIs, queries, and data structures.
So, while the backend of Squares functions right now, I already know there's many aspects of it that could be improved, refactored, or even removed. But, it's been a great learning experience and one that I wouldn't change.
Time management and scale of an MVP
As mentioned at the top of this post, I want project and SaaS building (especially in public) to become a regular occurrence for me so this hackathon has been a blessing in teaching me how to plan, prioritise, and structure my time to deliver a project on a tight deadline.
In hindsight, I realise I way overshot what was needed for this MVP and tried to deliver more features that I necessarily needed to in it. They're not actually included in the MVP app right now as I removed them due to not being stable and breaking the app on page load. But, if you look in the code on GitHub, you'll see references to components not used and API routes never consumed, these are the features that didn't make it into the MVP and really shouldn't of been built or included.
Here is the original design of Squares to give an idea of the scale of features I was thinking of. To deliver it on time I had to scale it back a bit which isn't ideal but a smaller more stable app is better than a large broken one.
What this hackathon has taught me is to build small and deploy often. During this month, I did the opposite and built some huge features locally, only to deploy them late on and find out they break everything...
Going forward, I'm going to think about what is truly needed in the MVP of a product to make it functionable, usable, and most importantly fulfil the core purpose of it. Then, work on delivering just that and no more. After the MVP is released, I'll start looking at extra features that would be nice to have.
What Comes Next?
We're now getting towards the end of the post so don't worry there's not much more to go now. To round out the journey of Squares and my experience in this hackathon, I want to talk about where I want to take it next and what my plans are after this hackathon is finished.
Keep Calm, Develop On
After this hackathon is done, I'm going to keep working on Squares (although maybe after a short break to recharge 😅). My plans will be to fix the last few bugs in the MVP and finish off implementing the features that weren't stable enough to get into it. Most likely this will include a complete rewrite of the backend to a more thoroughly thought out and planned solution probably involving AWS as mentioned earlier.
Build, Build, Build 🔥
Outside of Squares, I plan to build more projects using Next.js, Prisma, and PlanetScale. This tech stack has been absolutely amazing to work with and I don't have any complaints from my month of using it and would gladly use it again for future projects. So, keep your eyes peeled for future projects and products I build. 🚀
Closing Thoughts
July has been one hell of a journey, going from just an idea at the start of the month to a finished MVP of a product at the end of it. Along the way, I've learned so much, not just about PlanetScale but about so many other things like monorepos, the Vercel API, CI, and more. I hope I've given you an insight into the journey I've been on this month and some of the things I've learned.
If you have any questions or want to discuss something I've put in this post, drop a comment below or ping me a tweet and I'd love to chat with you. Also, if you'd like to contribute to Squares you can find all the links below, I appreciate every contribution regardless how small it is.
Finally, if you have any feedback I would love to hear it and I hope you enjoyed reading this post and using Squares. 😊
Links
Here are all the links you need to get started with Squares. 👇