Why not slowly decompose the functionality of your software into modules, abstract those behind some kind of abstract interface, and then move those abstracted modules into microservices when the time comes? Just by slowly rebuilding components as you need to touch them you'll be able to maintain some velocity on feature development, be able to incrementally test your changes to the codebase, and actually find the real world logical groupings of functionality (rather than guessing on business domain like most SOAs do).
Full rewrites like this are a pain, high risk, and always take longer than expected (feature creep/deployment/etc)
Full rewrites like this are a pain, high risk, and always take longer than expected (feature creep/deployment/etc)
This is accepted wisdom in large parts of the software development world, but I have become sceptical about treating it as a universal rule.
In the end, all software maintenance means rewriting part of the system at some scale, whether it’s a one-line bug fix or a Big Rewrite of the whole world. Both the risks and the potential benefits typically increase with the scale of the job, so how much of the system to change in one go is always a cost/benefit question.
All software maintenance where we delete or modify old code risks losing subtle bug fixes and behaviour tweaks that have accumulated in that code. However, it also gives us a chance to make the new code more systematic and to take into account additional knowledge gained and better tools that have become available since the older version was written. This means the new version also has an opportunity to eliminate subtle bugs and awkward edge cases that have accumulated in the old code.
For me, one of the times when a relatively large rewrite is indicated is when there are big changes to the foundations on which the system or some part of it is built: you need to support new hardware or run on a new operating system, there’s a big update to your programming language available and you want to be able to use it, the protocols or APIs or data models used to interact with other systems have changed. Another is when the original software architecture has been outgrown and technical debt is severely affecting productivity. In this case, it appears that both of those factors may apply, so maybe a wide-reaching update does make sense.
I'm of a similar mind but tend towards avoiding rewrites, mostly because I find refactoring more efficient.
And with that in mind, I should clarify im not opposed to a complete gutting of a legacy codebase and I'm not afraid of doing it in a full rewrite sense of the word either. I just like to have a strategy and keep master green throughout.
> risks losing subtle bug fixes and behaviour tweaks that have accumulated in that code.
This shouldn't be happening in a sufficiently tested or sufficiently understood system and I feel like using it as an excuse to not rewrite is a lack of reasoning skills.
Full rewrites are a waste of time because you end up spending way more time than expected making useless decisions around the edges about APIs that really don't matter that much.
Or you end up finding out your dream design has a fatal flaw. Or you end up just porting a bunch of boiler plate.
Rule of thumb,
If you can't figure out a peicemeal way to refactor what you have into what you want you want, you have no business discussing what a rewrite should entail.
And a caveat to all this, I know a couple of teams that rewrite every system on a two year schedule. They do this as a a way to stay nimble. I believe their feature set and domain requirements are well known. This sort of setup has a few benefits and traits I like. Most notably the fact design skills are constantly being flexed instead of stagnating. Junior devs are made into seniors with this process.
I'm of a similar mind but tend towards avoiding rewrites, mostly because I find refactoring more efficient.
Interesting choice of phrase. What would you say is the difference between refactoring and rewriting, assuming we’re talking about keeping the externally observable behaviour of the affected part of the system the same in each case?
This shouldn't be happening in a sufficiently tested or sufficiently understood system
Perhaps, but as systems and the teams that develop them evolve over time, it is all but inevitable that some knowledge and understanding are lost. You’ll never have perfect documentation. You’ll never have tests for every possible case. You’ll never have an unambiguous and readily located comment for every subtle detail in the code that someone seven years ago spent a couple of hours thinking about before they put finger to keyboard, when three other people have since refactored that code, generalised its interface, and fixed a performance glitch in a dependency so some aspects of the original design became unnecessary. Preserving knowledge and understanding as well as possible is one of the challenges facing any team with a long-lived system.
Those losses are a form of technical debt, and as it accumulates, two things happen. First, the original code becomes harder to maintain and develop. Second, the original code becomes more risky to rewrite. So unfortunately, there’s a fairly high probability in practice that a part of a system that has become sufficiently difficult that a large-scale rewrite is being considered will also be a part of the system where the current development team have incomplete understanding of the current implementation.
If you can't figure out a peicemeal way to refactor what you have into what you want you want, you have no business discussing what a rewrite should entail.
But how do you do a piecemeal refactoring exercise if, say, you’re moving to a new programming language? I had to rewrite a large part of a relatively complicated web UI a few years ago, like for like. Although it had been working reasonably well to that point and the design was quite clean and had stood the test of time, it was built around a Java applet. That design made sense when we were first implementing that system, at a time when JS was slow and many features we take for granted in browsers today weren’t yet widely supported. It was less helpful when the great and the good of the browser world decided that Java applets were a liability and they were going to make them harder and harder to use until eventually they killed them off completely. Now, we did have a clean, careful interface between the applet and the various other parts of the page that were written in JS, so we only had to redo that part of the system. However, we had no choice but to reimplement everything the applet did, using alternatives like JS and SVG, starting from scratch. This is the sort of scenario I was talking about before when I mentioned changes in the foundations of a system.
Rewriting piecemeal is a great idea but extracting microservices adds a whole new set of problems that likely aren't worth the tradeoff, especially when your team is just you or only few other people. eg: Now your devops responsibilities aren't just deploying and keeping 1 app up and running, it's all of them.
I specifically mention this in reference to "when the time comes". Sometimes the devops work is worth it. If you're building something that's going to get a predictable load of 10k QPS and is only going to read from a DB and return some data it might be a good idea to break that out, run that in a serverless configuration, etc.
It's all trade offs and just choosing when you want to make them. If adding 10k QPS load to your monolithic app is going to cost more than the (TOOLS + DEPLOYMENT STRATEGY + INTEGRATION TESTING + ETC) then it's worth pulling it out.
I 100% agree it's all trade-offs. I think cost is important, but in that scenario it's going to cost them a lot more in time to split out and maintain several services. Devops was only one example. Their development speed will also likely slow down by a good margin as they now have to worry about inter-service communication and all that comes with it (keeping schemas aligned, making sure the services can actually talk to one another, etc.).
I think services are a useful tool but for most problems you can get incredibly far with a monolith. I think a solid approach is to identify where there's a bottleneck in your monolith and extract that instead of splitting the application up for the sake of having separate services.
> If you're building something that's going to get a predictable load of 10k QPS and is only going to read from a DB and return some data it might be a good idea to break that out, run that in a serverless configuration, etc.
Serverless costs you money. Running a low-volume db in the same VM you serve the site and the web api costs you zero.
It may be rewritten but it is an entirely new product. The new code does not replace old code. The old code continues to live and run on/with the older product. This reduces risk of breaking things but also increases costs due to running multiple versions. It's all trade-offs.
I am on the camp that big rewrites are more often than not a mistake. The real reason behind most rewrites is that writing code is easier than reading code. A system has grown to be too complex to understand, and instead of taking the time to study and understand it, you start writing it from scratch instead. At some point you end up with a new system of perhaps comparable complexity as the original. You now understand that system better, but give it enough time you will get rusty, or new devs will join that do not understand it. Rinse and repeat.
Also many devs underestimate the reasons why code has ended up being smelly. Unaccounted edge cases, legacy system support, time constraints. Who says you won't hit time constraints during the rewrite too?
Of course there are many good reasons for a rewrite too. Being more experienced and a better dev today than you were yesterday when you wrote it, is a great way to clean up whatever amateur mistakes you might have created. Or perhaps the system is inefficient and now you know how to improve it. But incremental updates, and defining the critical code path and making sure that is as robust as possible (eg I am sure you can increase system performance by magnitudes by changing only 10% of the most-often run code), are great ways to start.
Good luck, and I am looking forward lessons learnt too!
The worst is reading the code, understanding it, then re-writing it in $LANGUAGE. Then you have a (buggy) emulation of the original product.
I believe a re-write should be a new product. New accounts will use it, and old ones will be migrated to it. Chances are, the way you did things the first time around wasn't ideal (but you couldn't have known back then), so the process is improving and not just the code.
I've done various rewrites of solo and duo projects.
Once all is settled, I ultimately tend to agree that I should've probably upgraded what I originally had, Ship of Theseus style. But then again, I also wasn't doing that, and the allure of a rewrite is what energized me to do all the work in the first place.
But I recognize this blog post as the sort of giddy excitement that you use to convince yourself that it's going to help your customers and be worth your time. Just look at it: all upsides, no downsides.
An unfortunate truth of solo/duo entrepreneuring is that things get created and written quick and dirty. You simply don't have the time to do everything by the book, especially when you're nights/weekends.
Getting to the point when you have to do a massive refactor is, in a way, a mark of success because it means you're growing and have a good customer base.
You can and should build a big codebase (as far as early stage startups go) without doing things quick and dirty. The additional time is an investment that will save you far more time down the road.
this only applies when you "know" that what you're building will stick around. If you are quickly iterating on an MVP it's more likely that code you write is going to need to be totally revamped.
Acquiring tech debt in an early stage startup often results in never having to pay back that debt
Not true. In the case that your MVP is actually worth something, you don't want to be spending all your time paying off tech debt when you could be focusing on growth.
Would you rather have a well-designed flop, or a poorly-designed hit that you can't scale fast enough?
Remember Friendster from 2003? It's quite possible that it would have not lost to Tribe/Myspace/Facebook if it didn't take minutes to connect before eventually timing out.
There are things that are worse than wasted effort, and one of them is a hockey stick that slips right through your fingers. By the time the curve is inflecting, if you can't keep up, you're burning customers and inviting competition. If you have to rewrite at that point, you are going to lose to competitors with deeper pockets and no PR baggage.
> Would you rather have a well-designed flop, or a poorly-designed hit that you can't scale fast enough?
Poorly-designed hit any day. Was this a trick question? :-)
(Obv, I'd prefer a well-designed hit, but I'd rather move fast and validate my market with a hack, than spend too much time and energy building something good that'll never be used.)
Poorly-designed market-hit is usually easier and more straightforward to fix than a well-designed market-flop.
I'll take the market hit any day! (Anyone remember the fairly pervasive "fail whales" along Twitter's bumpy evolution? Did they negatively impact the end state of the company?)
I do remember those, and iirc it almost sank the company. In fact I think that it absolutely impacted Twitter negatively, because the way they got through it was with massive investment in engineering funded by venture capital. So now Twitter is overcapitalized and beholden to be a size that makes that investment worthwhile, and can't make lean decisions that would be better for the world while also being sustainable for itself.
Let's ask ourselves, as either stakeholders or users of Twitter or participants in the culture borne of Twitter's poorly-designed success: are we happy with Twitter's arc?
I would rather have a Basecamp level of success that was well-designed and sustainable. And you don't get there by putting out a pile of shit and hoping for a runaway hit and then scrambling to raise billions so you can backfill everything you neglected to think about in the first place. The only ones who want that are VCs and the hyper-ambitious entrepreneurs who would rather have a billion dollars than make the world a better place.
Assuming you're building the correct thing. Small/early start up mode is about finding product market fit. The additional time is a waste that might never be needed or used. That additional time may also sink the endeavor. Regardless, no sense in beating yourself up and worrying about 20/20 hindsight. Software development produces two things of great value. Working software and information. Know what your current primary production goal is.
Despite the title, it doesn't appear he'll be waiting for a complete rewrite. The article discusses replacing a number of subsystems. I'd be surprised if one could pull it off simultaneously. Much more likely they will be tackled sequentially.
This is similiar to refactoring, just at a larger scale.
It really is a complete rewrite - switching from WordPress to Laravel (no code being reworked really there ...) and then rewriting the backend Python scripts to be OOP (minimal code reuse)
Rewrites of subsystems, one at a time, is not as risky as a wholesale rewrite, which Joel warns about. Some parts may be harder/riskier than others of course. Python scripts to OOP might reuse a lot depending on how it was written.
We just did this with our mobile app, which was in Phone gap, and is now in React Native. It was a horribly long process, I would have liked to avoid but I don't think there was much choice.
We were able to purge unused features that had previously added complexity. Building a new app with 4yrs of knowledge from the previous app was enlightening. Now that we are through it, I feel the product is better for it, but damn it took too long.
User email accounts aren’t exactly designed to send procedural emails from your application. I would use something like Postmark or AWS SES to send emails from your application instead of just dropping in the SMTP credentials for your Rackspace email.
I've recently been using freshdesk and while it is nice like a lot of support platforms I feel like everything is ultra clicky and I'm constantly clicking not knowing what will happen or unexpected things change in the view. Its a thing with all support platforms it seems.
Also... please people can we move off medium so I can stop getting spammy stuff about joining and etc? Clearly this person is a professional but at this point medium seems like using facebook to post a professional blog or something where this / that.
On top of the annoying stuff on Medium... I feel like using it for SEO also means that it is being posted "at" me and not "for" me.
That is to say they post because they want me to visit their business site ... not really tell me a story.
Now for any given user (the OP here) I wouldn't accuse them of that and I'm sure they'd like me to visit their site, but also want to genuinely share... but overall I start getting a different feeling from Medium posts...
Reminds me of when I was getting started with web development. SO MANY basics articles out there that I'm pretty sure are just resume fodder written by people who aren't that much better than me, and really not written to share as much as post on their resume, that really kinda sucks.
> I’ve always hosted all of my email on my own servers just to save money.
> I will switch from self-hosting my email to $2.99/month email accounts at Rackspace.
I think this is really smart, assuming Rackspace does a good job of delivering mail. I don't think self-hosting email ever makes sense from a cost savings perspective.
What is you're time worth? A single hiccup and you've blown base $3 per month or heck $50 per month in cost savings.
I currently point the MX records of all my domains to my existing Fastmail account.
Of all the things in the rewrite which are essential to the need to rewrite, which would be feasible to replace later, and which fall in between? Minimize scope ruthlessly. It's always easy to expand scope when a core is working not the other way around. Even if the entire 2.0 system was complete the migration without impacting customers seems intricate. Staged infra updates FTW.
I'm very interested in the first version you wrote.
It just seems like a good example of an MVP. You got the site up in Wordpress fairly quickly, modified it using PHP to your liking (did you have to in order to implement features), and then some python to actually do some work on the back end?
Very unclear, but I think that would be of interest to this audience.
edit: Apparently the rest of his blog is dedicated to this info so in case you are reading this, are as curious as I was, and didn't think to click around:
GSuite: Best in class spam filtering. Mail clients are excellent. Integration with the rest of the Google ecosystem is wonderful. Google, as an auth provider, is fantastic. Downtime is nearly non-existent.
O365: Enterprise class mail tools, sub-par protection (spam and phishing will get through). Mail clients are fine. Even better if you're a Windows shop. O365, as an auth provider, is not as clean. Not as fault tolerant, but getting much better.
I personally wouldn't bet on rackspace being around long term and I wouldn't start today building email on their platform. We've dealt with rackspace for years, we're mostly off of it now.
I quite curious about your motivations and product, as I work on custom hotspot which is bigger enough that current offers are outside scope but also we have just the basic features and adding things like facebook login support and marketing info capture with the gpdr in mind are difficult to prioritize. Anyway, a full rewrite is always a complicated task as always are more things that what one thinks ahead, but good luck with it!
This Joel on Software article “Things You Should Never Do” is quite timeless and worth a good read. The first one is never to rewrite your software. Your just giving your competitors a free break.
Great post, and appreciate what he is saying. But in this case I'm kind of locked into WordPress and the plugins I'm using. It will be much easier to start over with Laravel Spark which will give me more control. From there, I agree that it should never be rewritten again, only refactored.
You can encapsulate your business logic on the WordPress PHP side into PHP classes first.
What you want to do is break your WordPress application into classes containing the business logic and turn WordPress into a dumb wrapper for those classes. At that point, you can move those new classes over to Laravel when you are ready to make the switch.
For example, you can switch your customer dashboard views over to be written in Laravel. You can use either nginx/apache to route, or have a base index.php file that either includes the WordPress index.php or the Laravel index.php based upon PHP code. Within your new Laravel routes, you will be boostrapping WordPress from Laravel and calling WordPress code from within Laravel.
For example the WordPress users. You could create a new class and interface to encapsulate all of the WordPress user code inside your WordPressUser class that implements MyAppUserInterface. Eventually, you will create a new class that implements that interface for Laravel once you are ready to switch out WordPress for user management and use Laravel. With that said, I would likely leave users to be the very last thing you change. And quite possibly, think about leaving WordPress around solely for user auth and management.
The same idea works for posts and other entities you are currently using within WordPress.
On the WordPress side you can implement hooks that can call methods on the Laravel side as you move through the transition.
Rewrites are so often deadly. Please put some serious thought into it. No one knows your business better than you do, so I'll just leave you with that friendly advice.
Also, writing this makes me chuckle as I can't even get a response from a single resume I've sent out over the last two weeks. Sigh...
Have you considered keeping your wordpress as a "frontend" and making API calls to a backend build in laravel? You could slowly migrate your code to a `/v2/` path on the same system. Then, once everything is moved into API calls on the `/v2` path you can just write a new frontend that talks to the new APIs you've built.
Maybe it was downvoted when you saw it, but it isn't now. Maybe it will be downvoted again next time someone sees it. Who knows? This is why it's a bad idea to complain about downvotes.
I enjoy seeing successful examples like your project. Many of the successful products started simple like you did, while there are tones of over-engineered MVPs that never launched.
Solo entrepreneur full-rewrites are almost always churn inducing, feature stagnation, time sucks.
I'd advocate working on improvements to the current platform, one-by-one. Focus on features that increase your bottom line until you're at a MRR where losing 30% of your customers will be worth the cost of improvements.
I wouldn't describe in such harsh words (although maybe that helps bring the point across?), but overall I agree. Several of the items look like they could be improved completely isolated from the rest (eg. the email issue).
Moving from Wordpress to Laravel means that the basic stack remains the same (PHP). All in all, this looks like something that ought to be possible without a rewrite.
I'd start with building a new front router that calls liberally into wordpress as a backend.
Once wordpress is only managed (and accessible!) by the front router, one by one hack up wordpress to call into Laravel Spark stuff (eg. the user database) instead of using its own infrastructure. The end result would be that there's no wordpress code being used anymore (so drop it), and the site is completely moved over.
But that way every step in between won't be a regression to users, and it's realistic to add the odd emergency feature addition to the platform without having to maintain it twice (in the old platform and the new).
Agreed. I've been slowly migrating away from my one-true-love, of Perl, because it is easier to deploy golang binaries.
During the course of that I've isolated small sections of my project, and migrated them over a piece at time. Instead of perl "doing stuff" it now writes out data to JSON, and a golang service consumes that and does the actual back-end work.
This lets me rewrite the trickiest bits first, and also provides a level of isolation and abstraction I can test thoroughly.
Already I'm feeling much more productive, and I have more confidence that I don't need to install specific versions of gems (some ruby stuff too) and "unsupported" perl modules.
This should be "HostiFi 2.0: Why I’m completely rewriting my $5,735 MRR SaaS". There is no reason to leave off the actual name of the thing that's being worked on.
Why not slowly decompose the functionality of your software into modules, abstract those behind some kind of abstract interface, and then move those abstracted modules into microservices when the time comes? Just by slowly rebuilding components as you need to touch them you'll be able to maintain some velocity on feature development, be able to incrementally test your changes to the codebase, and actually find the real world logical groupings of functionality (rather than guessing on business domain like most SOAs do).
Full rewrites like this are a pain, high risk, and always take longer than expected (feature creep/deployment/etc)
Full rewrites like this are a pain, high risk, and always take longer than expected (feature creep/deployment/etc)
This is accepted wisdom in large parts of the software development world, but I have become sceptical about treating it as a universal rule.
In the end, all software maintenance means rewriting part of the system at some scale, whether it’s a one-line bug fix or a Big Rewrite of the whole world. Both the risks and the potential benefits typically increase with the scale of the job, so how much of the system to change in one go is always a cost/benefit question.
All software maintenance where we delete or modify old code risks losing subtle bug fixes and behaviour tweaks that have accumulated in that code. However, it also gives us a chance to make the new code more systematic and to take into account additional knowledge gained and better tools that have become available since the older version was written. This means the new version also has an opportunity to eliminate subtle bugs and awkward edge cases that have accumulated in the old code.
For me, one of the times when a relatively large rewrite is indicated is when there are big changes to the foundations on which the system or some part of it is built: you need to support new hardware or run on a new operating system, there’s a big update to your programming language available and you want to be able to use it, the protocols or APIs or data models used to interact with other systems have changed. Another is when the original software architecture has been outgrown and technical debt is severely affecting productivity. In this case, it appears that both of those factors may apply, so maybe a wide-reaching update does make sense.
I'm of a similar mind but tend towards avoiding rewrites, mostly because I find refactoring more efficient.
And with that in mind, I should clarify im not opposed to a complete gutting of a legacy codebase and I'm not afraid of doing it in a full rewrite sense of the word either. I just like to have a strategy and keep master green throughout.
> risks losing subtle bug fixes and behaviour tweaks that have accumulated in that code.
This shouldn't be happening in a sufficiently tested or sufficiently understood system and I feel like using it as an excuse to not rewrite is a lack of reasoning skills.
Full rewrites are a waste of time because you end up spending way more time than expected making useless decisions around the edges about APIs that really don't matter that much.
Or you end up finding out your dream design has a fatal flaw. Or you end up just porting a bunch of boiler plate.
Rule of thumb,
If you can't figure out a peicemeal way to refactor what you have into what you want you want, you have no business discussing what a rewrite should entail.
And a caveat to all this, I know a couple of teams that rewrite every system on a two year schedule. They do this as a a way to stay nimble. I believe their feature set and domain requirements are well known. This sort of setup has a few benefits and traits I like. Most notably the fact design skills are constantly being flexed instead of stagnating. Junior devs are made into seniors with this process.
I'm of a similar mind but tend towards avoiding rewrites, mostly because I find refactoring more efficient.
Interesting choice of phrase. What would you say is the difference between refactoring and rewriting, assuming we’re talking about keeping the externally observable behaviour of the affected part of the system the same in each case?
This shouldn't be happening in a sufficiently tested or sufficiently understood system
Perhaps, but as systems and the teams that develop them evolve over time, it is all but inevitable that some knowledge and understanding are lost. You’ll never have perfect documentation. You’ll never have tests for every possible case. You’ll never have an unambiguous and readily located comment for every subtle detail in the code that someone seven years ago spent a couple of hours thinking about before they put finger to keyboard, when three other people have since refactored that code, generalised its interface, and fixed a performance glitch in a dependency so some aspects of the original design became unnecessary. Preserving knowledge and understanding as well as possible is one of the challenges facing any team with a long-lived system.
Those losses are a form of technical debt, and as it accumulates, two things happen. First, the original code becomes harder to maintain and develop. Second, the original code becomes more risky to rewrite. So unfortunately, there’s a fairly high probability in practice that a part of a system that has become sufficiently difficult that a large-scale rewrite is being considered will also be a part of the system where the current development team have incomplete understanding of the current implementation.
If you can't figure out a peicemeal way to refactor what you have into what you want you want, you have no business discussing what a rewrite should entail.
But how do you do a piecemeal refactoring exercise if, say, you’re moving to a new programming language? I had to rewrite a large part of a relatively complicated web UI a few years ago, like for like. Although it had been working reasonably well to that point and the design was quite clean and had stood the test of time, it was built around a Java applet. That design made sense when we were first implementing that system, at a time when JS was slow and many features we take for granted in browsers today weren’t yet widely supported. It was less helpful when the great and the good of the browser world decided that Java applets were a liability and they were going to make them harder and harder to use until eventually they killed them off completely. Now, we did have a clean, careful interface between the applet and the various other parts of the page that were written in JS, so we only had to redo that part of the system. However, we had no choice but to reimplement everything the applet did, using alternatives like JS and SVG, starting from scratch. This is the sort of scenario I was talking about before when I mentioned changes in the foundations of a system.
Rewriting piecemeal is a great idea but extracting microservices adds a whole new set of problems that likely aren't worth the tradeoff, especially when your team is just you or only few other people. eg: Now your devops responsibilities aren't just deploying and keeping 1 app up and running, it's all of them.
I specifically mention this in reference to "when the time comes". Sometimes the devops work is worth it. If you're building something that's going to get a predictable load of 10k QPS and is only going to read from a DB and return some data it might be a good idea to break that out, run that in a serverless configuration, etc.
It's all trade offs and just choosing when you want to make them. If adding 10k QPS load to your monolithic app is going to cost more than the (TOOLS + DEPLOYMENT STRATEGY + INTEGRATION TESTING + ETC) then it's worth pulling it out.
I 100% agree it's all trade-offs. I think cost is important, but in that scenario it's going to cost them a lot more in time to split out and maintain several services. Devops was only one example. Their development speed will also likely slow down by a good margin as they now have to worry about inter-service communication and all that comes with it (keeping schemas aligned, making sure the services can actually talk to one another, etc.).
I think services are a useful tool but for most problems you can get incredibly far with a monolith. I think a solid approach is to identify where there's a bottleneck in your monolith and extract that instead of splitting the application up for the sake of having separate services.
> If you're building something that's going to get a predictable load of 10k QPS and is only going to read from a DB and return some data it might be a good idea to break that out, run that in a serverless configuration, etc.
Serverless costs you money. Running a low-volume db in the same VM you serve the site and the web api costs you zero.
A classic on the dangers on full rewrites - https://www.joelonsoftware.com/2000/04/06/things-you-should-...
Not sure where I heard it (probably some podcast), but Basecamp is rewritten every few years from scratch.
It may be rewritten but it is an entirely new product. The new code does not replace old code. The old code continues to live and run on/with the older product. This reduces risk of breaking things but also increases costs due to running multiple versions. It's all trade-offs.
Sounds like a plan. Looking forward to reading the 'lessons learned' blog post in 6-12-24 months.
Touché and well played.
I am on the camp that big rewrites are more often than not a mistake. The real reason behind most rewrites is that writing code is easier than reading code. A system has grown to be too complex to understand, and instead of taking the time to study and understand it, you start writing it from scratch instead. At some point you end up with a new system of perhaps comparable complexity as the original. You now understand that system better, but give it enough time you will get rusty, or new devs will join that do not understand it. Rinse and repeat.
Also many devs underestimate the reasons why code has ended up being smelly. Unaccounted edge cases, legacy system support, time constraints. Who says you won't hit time constraints during the rewrite too?
Of course there are many good reasons for a rewrite too. Being more experienced and a better dev today than you were yesterday when you wrote it, is a great way to clean up whatever amateur mistakes you might have created. Or perhaps the system is inefficient and now you know how to improve it. But incremental updates, and defining the critical code path and making sure that is as robust as possible (eg I am sure you can increase system performance by magnitudes by changing only 10% of the most-often run code), are great ways to start.
Good luck, and I am looking forward lessons learnt too!
The worst is reading the code, understanding it, then re-writing it in $LANGUAGE. Then you have a (buggy) emulation of the original product.
I believe a re-write should be a new product. New accounts will use it, and old ones will be migrated to it. Chances are, the way you did things the first time around wasn't ideal (but you couldn't have known back then), so the process is improving and not just the code.
I've done various rewrites of solo and duo projects.
Once all is settled, I ultimately tend to agree that I should've probably upgraded what I originally had, Ship of Theseus style. But then again, I also wasn't doing that, and the allure of a rewrite is what energized me to do all the work in the first place.
But I recognize this blog post as the sort of giddy excitement that you use to convince yourself that it's going to help your customers and be worth your time. Just look at it: all upsides, no downsides.
Tread carefully, OP.
An unfortunate truth of solo/duo entrepreneuring is that things get created and written quick and dirty. You simply don't have the time to do everything by the book, especially when you're nights/weekends.
Getting to the point when you have to do a massive refactor is, in a way, a mark of success because it means you're growing and have a good customer base.
You can and should build a big codebase (as far as early stage startups go) without doing things quick and dirty. The additional time is an investment that will save you far more time down the road.
this only applies when you "know" that what you're building will stick around. If you are quickly iterating on an MVP it's more likely that code you write is going to need to be totally revamped.
Acquiring tech debt in an early stage startup often results in never having to pay back that debt
Not true. In the case that your MVP is actually worth something, you don't want to be spending all your time paying off tech debt when you could be focusing on growth.
Quick honest question, what's your exposure and familiarity to finance?
But what if the product is a flop? Part of the benefit of "quick and dirty" is that you can quickly verify the market and iterate on product fit.
Would you rather have a well-designed flop, or a poorly-designed hit that you can't scale fast enough?
Remember Friendster from 2003? It's quite possible that it would have not lost to Tribe/Myspace/Facebook if it didn't take minutes to connect before eventually timing out.
There are things that are worse than wasted effort, and one of them is a hockey stick that slips right through your fingers. By the time the curve is inflecting, if you can't keep up, you're burning customers and inviting competition. If you have to rewrite at that point, you are going to lose to competitors with deeper pockets and no PR baggage.
> Would you rather have a well-designed flop, or a poorly-designed hit that you can't scale fast enough?
Poorly-designed hit any day. Was this a trick question? :-)
(Obv, I'd prefer a well-designed hit, but I'd rather move fast and validate my market with a hack, than spend too much time and energy building something good that'll never be used.)
Poorly-designed market-hit is usually easier and more straightforward to fix than a well-designed market-flop.
I'll take the market hit any day! (Anyone remember the fairly pervasive "fail whales" along Twitter's bumpy evolution? Did they negatively impact the end state of the company?)
I do remember those, and iirc it almost sank the company. In fact I think that it absolutely impacted Twitter negatively, because the way they got through it was with massive investment in engineering funded by venture capital. So now Twitter is overcapitalized and beholden to be a size that makes that investment worthwhile, and can't make lean decisions that would be better for the world while also being sustainable for itself.
Let's ask ourselves, as either stakeholders or users of Twitter or participants in the culture borne of Twitter's poorly-designed success: are we happy with Twitter's arc?
I would rather have a Basecamp level of success that was well-designed and sustainable. And you don't get there by putting out a pile of shit and hoping for a runaway hit and then scrambling to raise billions so you can backfill everything you neglected to think about in the first place. The only ones who want that are VCs and the hyper-ambitious entrepreneurs who would rather have a billion dollars than make the world a better place.
Assuming you're building the correct thing. Small/early start up mode is about finding product market fit. The additional time is a waste that might never be needed or used. That additional time may also sink the endeavor. Regardless, no sense in beating yourself up and worrying about 20/20 hindsight. Software development produces two things of great value. Working software and information. Know what your current primary production goal is.
Despite the title, it doesn't appear he'll be waiting for a complete rewrite. The article discusses replacing a number of subsystems. I'd be surprised if one could pull it off simultaneously. Much more likely they will be tackled sequentially.
This is similiar to refactoring, just at a larger scale.
It really is a complete rewrite - switching from WordPress to Laravel (no code being reworked really there ...) and then rewriting the backend Python scripts to be OOP (minimal code reuse)
Rewrites of subsystems, one at a time, is not as risky as a wholesale rewrite, which Joel warns about. Some parts may be harder/riskier than others of course. Python scripts to OOP might reuse a lot depending on how it was written.
We just did this with our mobile app, which was in Phone gap, and is now in React Native. It was a horribly long process, I would have liked to avoid but I don't think there was much choice.
We were able to purge unused features that had previously added complexity. Building a new app with 4yrs of knowledge from the previous app was enlightening. Now that we are through it, I feel the product is better for it, but damn it took too long.
Where could you have been had you focused on cleaning up your existing codebase?
Debated that a lot, but it was determined that Phone Gap had no future
User email accounts aren’t exactly designed to send procedural emails from your application. I would use something like Postmark or AWS SES to send emails from your application instead of just dropping in the SMTP credentials for your Rackspace email.
I've recently been using freshdesk and while it is nice like a lot of support platforms I feel like everything is ultra clicky and I'm constantly clicking not knowing what will happen or unexpected things change in the view. Its a thing with all support platforms it seems.
Also... please people can we move off medium so I can stop getting spammy stuff about joining and etc? Clearly this person is a professional but at this point medium seems like using facebook to post a professional blog or something where this / that.
I'm actually looking at using Intercom now instead of FreshDesk, FreshChat, and Drip.
Sorry about Medium lol I'm just lazy. I plan to get around to hosting my own blog on Ghost.
No problem, I get why people use it, it's just grown.. less than a desirable place to read things.
Using Medium is good for SEO.
It's one reason why people still use it.
On top of the annoying stuff on Medium... I feel like using it for SEO also means that it is being posted "at" me and not "for" me.
That is to say they post because they want me to visit their business site ... not really tell me a story.
Now for any given user (the OP here) I wouldn't accuse them of that and I'm sure they'd like me to visit their site, but also want to genuinely share... but overall I start getting a different feeling from Medium posts...
Reminds me of when I was getting started with web development. SO MANY basics articles out there that I'm pretty sure are just resume fodder written by people who aren't that much better than me, and really not written to share as much as post on their resume, that really kinda sucks.
> I’ve always hosted all of my email on my own servers just to save money. > I will switch from self-hosting my email to $2.99/month email accounts at Rackspace.
I think this is really smart, assuming Rackspace does a good job of delivering mail. I don't think self-hosting email ever makes sense from a cost savings perspective.
What is you're time worth? A single hiccup and you've blown base $3 per month or heck $50 per month in cost savings.
I currently point the MX records of all my domains to my existing Fastmail account.
Of all the things in the rewrite which are essential to the need to rewrite, which would be feasible to replace later, and which fall in between? Minimize scope ruthlessly. It's always easy to expand scope when a core is working not the other way around. Even if the entire 2.0 system was complete the migration without impacting customers seems intricate. Staged infra updates FTW.
I'm very interested in the first version you wrote.
It just seems like a good example of an MVP. You got the site up in Wordpress fairly quickly, modified it using PHP to your liking (did you have to in order to implement features), and then some python to actually do some work on the back end?
Very unclear, but I think that would be of interest to this audience.
edit: Apparently the rest of his blog is dedicated to this info so in case you are reading this, are as curious as I was, and didn't think to click around:
https://medium.com/@reillychase
> I will switch from self-hosting my email to $2.99/month email accounts at Rackspace.
As someone who has spent the last year or so migrating consulting clients off of Rackspace services - I have to ask... Why Rackspace email?
No reason really. What would you suggest?
Not OP, but my $0.02... GSuite or O365.
GSuite: Best in class spam filtering. Mail clients are excellent. Integration with the rest of the Google ecosystem is wonderful. Google, as an auth provider, is fantastic. Downtime is nearly non-existent.
O365: Enterprise class mail tools, sub-par protection (spam and phishing will get through). Mail clients are fine. Even better if you're a Windows shop. O365, as an auth provider, is not as clean. Not as fault tolerant, but getting much better.
I wonder— at a time in the future when every email in the country passes through a Google server, will they finally bring back Wave?
I really like zohomail suite. Pretty similar to gmail, I pay like $12/year for the lite plan.
I personally wouldn't bet on rackspace being around long term and I wouldn't start today building email on their platform. We've dealt with rackspace for years, we're mostly off of it now.
Good to know. Recommendation instead?
I like mailgun but I only use their free tier.
Mailgun is owned by rackspace. https://en.wikipedia.org/wiki/Rackspace#Acquisitions
Correction... Was. It's an independent company now. https://www.mailgun.com/blog/mailgun-becomes-an-independent-...
I quite curious about your motivations and product, as I work on custom hotspot which is bigger enough that current offers are outside scope but also we have just the basic features and adding things like facebook login support and marketing info capture with the gpdr in mind are difficult to prioritize. Anyway, a full rewrite is always a complicated task as always are more things that what one thinks ahead, but good luck with it!
This Joel on Software article “Things You Should Never Do” is quite timeless and worth a good read. The first one is never to rewrite your software. Your just giving your competitors a free break.
https://www.joelonsoftware.com/2000/04/06/things-you-should-...
Joel’s conclusions were based on a single data point. There were successful rewrites as well: https://medium.com/@herbcaudill/lessons-from-6-software-rewr...
In particular, Basecamp successful rewrite seems more applicable here, since both are web services. They never sunset previous two versions though.
Edit: typo.
Actually it was based on multiple datapoints: Netscape, Quattro Pro, Arago, and Microsoft Word.
I rewrote my app from AngularJS to Angular 2, very happy with result. Rewrite is justified if it will allow to move faster in the future.
This! OP, please consider thinking over this before you set out on the voyage.
Great post, and appreciate what he is saying. But in this case I'm kind of locked into WordPress and the plugins I'm using. It will be much easier to start over with Laravel Spark which will give me more control. From there, I agree that it should never be rewritten again, only refactored.
You can encapsulate your business logic on the WordPress PHP side into PHP classes first.
What you want to do is break your WordPress application into classes containing the business logic and turn WordPress into a dumb wrapper for those classes. At that point, you can move those new classes over to Laravel when you are ready to make the switch.
For example, you can switch your customer dashboard views over to be written in Laravel. You can use either nginx/apache to route, or have a base index.php file that either includes the WordPress index.php or the Laravel index.php based upon PHP code. Within your new Laravel routes, you will be boostrapping WordPress from Laravel and calling WordPress code from within Laravel.
For example the WordPress users. You could create a new class and interface to encapsulate all of the WordPress user code inside your WordPressUser class that implements MyAppUserInterface. Eventually, you will create a new class that implements that interface for Laravel once you are ready to switch out WordPress for user management and use Laravel. With that said, I would likely leave users to be the very last thing you change. And quite possibly, think about leaving WordPress around solely for user auth and management.
The same idea works for posts and other entities you are currently using within WordPress.
On the WordPress side you can implement hooks that can call methods on the Laravel side as you move through the transition.
Rewrites are so often deadly. Please put some serious thought into it. No one knows your business better than you do, so I'll just leave you with that friendly advice.
Also, writing this makes me chuckle as I can't even get a response from a single resume I've sent out over the last two weeks. Sigh...
Have you considered keeping your wordpress as a "frontend" and making API calls to a backend build in laravel? You could slowly migrate your code to a `/v2/` path on the same system. Then, once everything is moved into API calls on the `/v2` path you can just write a new frontend that talks to the new APIs you've built.
Not sure why this was downvoted. This is truth, and applies entirely to the original post.
Maybe it was downvoted when you saw it, but it isn't now. Maybe it will be downvoted again next time someone sees it. Who knows? This is why it's a bad idea to complain about downvotes.
It was down-voted because I didn't explain why the article was relevant. I added the last two sentences a bit later.
I enjoy seeing successful examples like your project. Many of the successful products started simple like you did, while there are tones of over-engineered MVPs that never launched.
Cheers!
Cringe.
Solo entrepreneur full-rewrites are almost always churn inducing, feature stagnation, time sucks.
I'd advocate working on improvements to the current platform, one-by-one. Focus on features that increase your bottom line until you're at a MRR where losing 30% of your customers will be worth the cost of improvements.
I wouldn't describe in such harsh words (although maybe that helps bring the point across?), but overall I agree. Several of the items look like they could be improved completely isolated from the rest (eg. the email issue).
Moving from Wordpress to Laravel means that the basic stack remains the same (PHP). All in all, this looks like something that ought to be possible without a rewrite.
I'd start with building a new front router that calls liberally into wordpress as a backend. Once wordpress is only managed (and accessible!) by the front router, one by one hack up wordpress to call into Laravel Spark stuff (eg. the user database) instead of using its own infrastructure. The end result would be that there's no wordpress code being used anymore (so drop it), and the site is completely moved over.
But that way every step in between won't be a regression to users, and it's realistic to add the odd emergency feature addition to the platform without having to maintain it twice (in the old platform and the new).
> Cringe.
Please don't do that here. It breaks the site guidelines: https://news.ycombinator.com/newsguidelines.html
Agreed. I've been slowly migrating away from my one-true-love, of Perl, because it is easier to deploy golang binaries.
During the course of that I've isolated small sections of my project, and migrated them over a piece at time. Instead of perl "doing stuff" it now writes out data to JSON, and a golang service consumes that and does the actual back-end work.
This lets me rewrite the trickiest bits first, and also provides a level of isolation and abstraction I can test thoroughly.
Already I'm feeling much more productive, and I have more confidence that I don't need to install specific versions of gems (some ruby stuff too) and "unsupported" perl modules.
Rackspace email? Why not gsuite?
This should be "HostiFi 2.0: Why I’m completely rewriting my $5,735 MRR SaaS". There is no reason to leave off the actual name of the thing that's being worked on.