If you're using Make as a command runner or a "standard entry point" to a project, instead of using it as a build system that tracks dependencies between files, I highly recommend using `just` instead: https://github.com/casey/just
It has this functionality built-in, and avoids a lot of Make's idiosyncrasies. (Not affiliated, just a fan.)
Make is fiddly, has gotchas and is far from perfect, from the development side - but it's certainly "good enough", and it's great once you've got your Makefile done. And it's available, or at least easily attainable, pretty much everywhere. I even use it on Windows.
Yeah, but if you follow the advice FTA, you now have to rely on a usable python version and make. Still fairly common, but what if I use node instead? I think the point is -- the more usability you add the more you potentially dilute how ubiquitous it is. I think it's okay to point to new tools in the space that shake things up and add usability for the purposes Make currently is used for. Ubiquity doesn't happen over night.
Even if I followed the advice in the article (I don't really plan to), python3 is very likely to already be there in a Linux distro for other reasons, whereas node.js is very likely to not be there.
If you're on another OS where neither make nor python3 nor node.js are there by default, then sure you're already going to manually install make, so the choice of python3 or node.js is basically identical.
Also, if you add a new target that would benefit from tracking dependencies between files, then make is already ready to do that.
I use makefiles for my Rust code with just default+test+install targets that do no dependency tracking (since Rust's build system already does that much better). But if I need to add a target for, say, validating some OpenAPI spec that only needs to run if the spec file updates, then that can go in the same Makefile.
I wanted to use Just at one point, but was disappointed that it doesn't support parallel execution. I ended up making a quick tool to convert a ZSH file into a Ninja file to take advantage of its pool feature.
> GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program's source files.
What is idiosyncratic is to think Make is first a command runner, and only after a file generator, and complain that it is doing a poor job at the former when it clearly says it is focusing on the latter.
I read the "idiosyncrasies" section on the just README and they are all performance related targeting file generation. All of those can be learned in one minute, as part of "I'll always find Makefiles in the wild, it's good to know the basics".
For historical reasons my team uses Makefiles for executing data science workflows[1]. They are auto-documenting which makes it not just helpful on me when I forget an underused target, but it's perhaps even more useful when onboarding new team members. Rather than directing them to some out of band docs, it's convenient to simply direct them to use make to get the help they need[2].
1. There are reasons, like consistency of interface which I value even more than autodocumentation, but that is outside the scope of this discussion.
For data science specifically, I would strongly suggest looking into DVC: https://dvc.org/.
You can easily write DVC stage files by hand as a straightforward Makefile replacement, and integrate other features into your workflow as needed/desired.
PS: quite feature complete but not yet well marketed so to speak. I'm actually recording an asciinema session this week in order for a visitor to grasp quicker the mentioned benefits.
Which is also included in your excellent GNU Make Book¹, which you should've pimped directly ;) It is both a fantastic general guide and an amazing source of usable little recipes, thanks!
A bit reluctant to share this as it is old (2005?) and would not support features I tend to use now. But if you are just starting,
or stick to the most basic functionality it may be of use as a way to visualize the execution graph (DAG)
I've got a similar setup but just
This was from HN a couple years ago
I've been on HN for years, but missed this. Thank you!
Ah yes, summon Cthulhu to explain the makefile to you. Seems extreme to me but some people have SAN to spare.
Yeah I ended up with my own variant from here as well many moons ago:
I’ve been doing that as well. Here [1] a more elaborate example, might be the original article.
[1] https://www.thapaliya.com/en/writings/well-documented-makefi...
If you're using Make as a command runner or a "standard entry point" to a project, instead of using it as a build system that tracks dependencies between files, I highly recommend using `just` instead: https://github.com/casey/just
It has this functionality built-in, and avoids a lot of Make's idiosyncrasies. (Not affiliated, just a fan.)
I think part of the appeal of make as a script runner is the ubiquity.
Was going to say the same thing.
Make is fiddly, has gotchas and is far from perfect, from the development side - but it's certainly "good enough", and it's great once you've got your Makefile done. And it's available, or at least easily attainable, pretty much everywhere. I even use it on Windows.
Yeah, but if you follow the advice FTA, you now have to rely on a usable python version and make. Still fairly common, but what if I use node instead? I think the point is -- the more usability you add the more you potentially dilute how ubiquitous it is. I think it's okay to point to new tools in the space that shake things up and add usability for the purposes Make currently is used for. Ubiquity doesn't happen over night.
Even if I followed the advice in the article (I don't really plan to), python3 is very likely to already be there in a Linux distro for other reasons, whereas node.js is very likely to not be there.
If you're on another OS where neither make nor python3 nor node.js are there by default, then sure you're already going to manually install make, so the choice of python3 or node.js is basically identical.
Multiple comments show how to do this with an awk, grep or sed one liner in the Makefile.
Another part might be parallel execution, running multiple shells at the same time
https://github.com/rofl0r/jobflow
Also, if you add a new target that would benefit from tracking dependencies between files, then make is already ready to do that.
I use makefiles for my Rust code with just default+test+install targets that do no dependency tracking (since Rust's build system already does that much better). But if I need to add a target for, say, validating some OpenAPI spec that only needs to run if the spec file updates, then that can go in the same Makefile.
I wanted to use Just at one point, but was disappointed that it doesn't support parallel execution. I ended up making a quick tool to convert a ZSH file into a Ninja file to take advantage of its pool feature.
From GNU Make webpage:
> GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program's source files.
What is idiosyncratic is to think Make is first a command runner, and only after a file generator, and complain that it is doing a poor job at the former when it clearly says it is focusing on the latter.
I read the "idiosyncrasies" section on the just README and they are all performance related targeting file generation. All of those can be learned in one minute, as part of "I'll always find Makefiles in the wild, it's good to know the basics".
For historical reasons my team uses Makefiles for executing data science workflows[1]. They are auto-documenting which makes it not just helpful on me when I forget an underused target, but it's perhaps even more useful when onboarding new team members. Rather than directing them to some out of band docs, it's convenient to simply direct them to use make to get the help they need[2].
1. There are reasons, like consistency of interface which I value even more than autodocumentation, but that is outside the scope of this discussion.
2. For small orders of help.
For data science specifically, I would strongly suggest looking into DVC: https://dvc.org/.
You can easily write DVC stage files by hand as a straightforward Makefile replacement, and integrate other features into your workflow as needed/desired.
This is exactly my experience which lead me to create https://github.com/ysoftwareab/yplatform - with a consistent make interface https://github.com/ysoftwareab/yplatform/tree/master/build.m...
PS: quite feature complete but not yet well marketed so to speak. I'm actually recording an asciinema session this week in order for a visitor to grasp quicker the mentioned benefits.
Native GNU Make way to do this: https://www.cmcrossroads.com/article/self-documenting-makefi...
Which is also included in your excellent GNU Make Book¹, which you should've pimped directly ;) It is both a fantastic general guide and an amazing source of usable little recipes, thanks!
¹ https://nostarch.com/gnumake
I try not to pimp my book. It’s unseemly :-)
@sed -ne '/@sed/!s/## //p' $(MAKEFILE_LIST)
We use this strategy at work: https://www.thapaliya.com/en/writings/well-documented-makefi...
It has sections and categorization.
I've been using this (different snippet though) for about 6 years now. It's great. "Poor man's CLI" is what I call my makefiles. :D Example: https://github.com/denibertovic/makefiles/blob/master/bare.m...
:thumbsup: This one allows you to put the '## help text' comments on the line just before the target. No need to put them after the prerequisite.
A bit reluctant to share this as it is old (2005?) and would not support features I tend to use now. But if you are just starting, or stick to the most basic functionality it may be of use as a way to visualize the execution graph (DAG)
https://github.com/TomConlin/MakefileViz
Here's my starter Makefile with colored help : https://www.bbkane.com/blog/a-fancy-makefile/
I don't use make often, so it also contains beginner friendly comments about the syntax
Throwing mine into the mix:
What I find interesting is programmers who are very scrupulous about documenting their code completely fail to document makefiles.
I have approached the problem in a similar way, but without python.
I published my self-documenting docker Makefiles here:
https://github.com/Infused-Insight/docker_makefiles
Clever! I like that....
(Now if I could integrate that into the autotools we use for some of our stuff....)
Babashka tasks has a :doc option for this:
https://book.babashka.org/#_discoverability
Feels like something Make would benefit to have out of the box.