It's interesting to see how many focus on the nested, recursive representations of expression and statement syntax.
It's understandable for teaching the concepts of syntax to people learning programming for the first time, but I don't think it tackles any of the essential complexity, or what makes programming hard.
It keeps getting reinvented in different guises, often with an air of democratizing coding, taking it out of the hands of the high priests (IT) and into the hands of the people. But what's hard about programming isn't if-statements, loops and assignments. It's incorrect assumptions and broken invariants, usually in runtime data, generated by code which is often distant.
Relatively few interfaces seem to deal with manipulating data rather than code. I think working with data is more likely to be useful and successful. Look at things like Visual Basic - a form designer is effectively manipulating a stored data structure which defines how a runtime data structure is going to get constructed.
So I wrote a projectional/structure editor[1] that does this sort of nested tiles representation. The reason for it wasn't to teach syntax though, it was to provide users with bigger, easier to interact with UI elements (reasons below).
I wanted the basic elements of the interface to be at a higher level than individual characters. The nested squares style rendering is highlighting grammatical structures in the language—not to teach syntax, but just to indicate where an interactable UI element is, the same way you have rectangular bounds around a button etc.
My motivation for this was to make it easier to manipulate code with lower-precision input devices (e.g. the microsoft kinect / leap motion), but I think there is general interest because a lot of the people writing these editors start from the premise that interacting with the AST directly (as opposed to character sequence) opens up doors, and nested squares is just kinda the most natural/obvious way of rendering an AST. It's also a more natural pairing with the input style of many of these interfaces (e.g. Lamdu[2] or Fructure[3]) which also operate at a higher level than character sequences.
The alternate rendering in itself isn't world-changing, it's the underlying change to directly manipulating a model that then gets rendered (as opposed to parsing the interface—i.e. character sequences—in order to arrive at the model) that's important.
(And even it probably isn't the most important revision to programming tools in general—but that doesn't make it not worth working on if someone's interest is in that direction.)
> I think working with data is more likely to be useful and successful. Look at things like Visual Basic - a form designer is effectively manipulating a stored data structure which defines how a runtime data structure is going to get constructed.
I think data manipulation interfaces are a good line of reasoning to investigate, but form designers aren't that, they're just for controlling visual placement of controls. The real MVP of dataflow manipulation programming for non-programmers is Excel.
Form designers are editing a data structure. It's not just visual placement of controls; it's the settings on those controls: content, appearance, behaviour, event bindings. And not just controls, also components.
VB and Delphi have components, non-visual controls that live on design surfaces, where their properties can be edited. Things like database connections and table connectors, which can further be linked up with data aware controls. Whole simple form-based CRUD applications can be built without any code at all, merely by wiring together components.
Some of these interfaces might be really fun to implement, but there are very few of them that I think I'd want to use.
For me, writing code is about the easiest thing I do. Other than identifier completion, I don't need much beyond a basic text editor.
On the other hand, understanding code -- not just exploring it, but understanding it -- is the the hardest thing I have to do, and that's where a good UI could be helpful. I only made a superficial scan, but it didn't really seem like any of these UIs were designed to do that.
I don't really need a map to a code base so much as I need a tool for making my own maps (not just one!) into the codebase as I try to understand it. Or understand at least enough of it to do my current task.
Phenomenal compendium, I love it. There always have been a lot of visual approaches to programming, but only a few had a relative success, and always restricted to a very limited domain. There is no competition to raw text editing when dealing with big, general problems and projects (yet). Why? Information density, convenience and fast editing may be the main factors, imho..
I think a more likely reason is keyboards. The main input device we use to interact with computers is a typewriter descendant, optimized for typing text. Mice are flexible but keyboards allow for more complex input.
Touch devices (like touchscreens or touchpads) could have helped with more direct manipulation of visual elements but they are always used on very limited (in terms of functionality and form factor) systems. On desktop computers they are far from ubiquitous and 99% of the time they just act as subpar mice.
I've come to think that one of the other problems non-textual approaches need to solve to win at scale is that they need to somehow get rid of their coordinates. Right now, most of the non-text oriented approaches I see mean that I have to not only understand the same thing I'd have to understand in a textual representation, but I also have the additional complexity of all of my code having coordinates in a 2D space, and then if you start drawing lines on the screen for relationships, additional accidental complexity in the state of the lines too.
The more I think about it, the more I think the people who tend to make criticisms about textual representations and then argue in favor of visual representations are not just wrong, but usually have things actively backwards; almost every criticism they make about text is actually a positive aspect of it. Not having a literal line appear from a call site to what it is calling, for instance, is a net positive, not a problem. I think this is why visual stuff has been bubbling slowly for decades and just never seems to get anywhere the way its advocates hope.
This must be a big part of it. Another is the kinds of computing environments that the majority of users, developers, and even CS researchers live and work inside of. For there to be truly new UIs the question of "what is symbolic representation in the medium of computing" (that is beyond written text) needs to be taken seriously, and that's just not going to happen when everything is Unix
It's not that clear cut. Large portion of programming activity already involves keyboard shortcuts etc rather than straight out typing code. I'd expect any good non-text based code editor to still be fully keyboard driven, or at least as much as current-day editors
Maybe the solution is that we need to get away from general purpose languages for most software. There could be very powerful visual languages that one could develop given a specific domain. I work in pharma and I see a ton of scope for this, especially for the automation of daily work. Scientists might not be the best with C but they could be amazing given a DSL in a domain that they completely grok. I've begun working more and more with Siemens' S7 GRAPH language for sequential process control and its bloody amazing how it lets me visually represent complicated state transitions, loops and conditional logic in a very easy to understand intuitive way. It also lets you establish pre/post invariants, etc. I used to scoff at PLCs thinking I could trivially write than in C, but even ladder logic has humbled me more than a few times.
I think one reason is self-selection bias: Because all programming languages are text, everyone who is a programmer is necessarily adept, comfortable, and intimately familiar with representing programs as text.
There is a whole other kind of programmer which doesn't exist yet— a visual coder— all of whom are doing something else right now, because there are no tools for them. It takes a large leap of imagination for a text-coder to empathize with these still non-existent visual-coders and make the tool that they need, and a large leap of imagination to envision a complete visual coding system from the ground up which can be used to solve actual problems and compete with the very mature text code tools that are already out there.
It's interesting to see how many focus on the nested, recursive representations of expression and statement syntax.
It's understandable for teaching the concepts of syntax to people learning programming for the first time, but I don't think it tackles any of the essential complexity, or what makes programming hard.
It keeps getting reinvented in different guises, often with an air of democratizing coding, taking it out of the hands of the high priests (IT) and into the hands of the people. But what's hard about programming isn't if-statements, loops and assignments. It's incorrect assumptions and broken invariants, usually in runtime data, generated by code which is often distant.
Relatively few interfaces seem to deal with manipulating data rather than code. I think working with data is more likely to be useful and successful. Look at things like Visual Basic - a form designer is effectively manipulating a stored data structure which defines how a runtime data structure is going to get constructed.
So I wrote a projectional/structure editor[1] that does this sort of nested tiles representation. The reason for it wasn't to teach syntax though, it was to provide users with bigger, easier to interact with UI elements (reasons below).
I wanted the basic elements of the interface to be at a higher level than individual characters. The nested squares style rendering is highlighting grammatical structures in the language—not to teach syntax, but just to indicate where an interactable UI element is, the same way you have rectangular bounds around a button etc.
My motivation for this was to make it easier to manipulate code with lower-precision input devices (e.g. the microsoft kinect / leap motion), but I think there is general interest because a lot of the people writing these editors start from the premise that interacting with the AST directly (as opposed to character sequence) opens up doors, and nested squares is just kinda the most natural/obvious way of rendering an AST. It's also a more natural pairing with the input style of many of these interfaces (e.g. Lamdu[2] or Fructure[3]) which also operate at a higher level than character sequences.
The alternate rendering in itself isn't world-changing, it's the underlying change to directly manipulating a model that then gets rendered (as opposed to parsing the interface—i.e. character sequences—in order to arrive at the model) that's important.
(And even it probably isn't the most important revision to programming tools in general—but that doesn't make it not worth working on if someone's interest is in that direction.)
[1] http://symbolflux.com/projects/tiledtext
[2] https://www.lamdu.org/
[3] https://github.com/disconcision/fructure
> I think working with data is more likely to be useful and successful. Look at things like Visual Basic - a form designer is effectively manipulating a stored data structure which defines how a runtime data structure is going to get constructed.
I think data manipulation interfaces are a good line of reasoning to investigate, but form designers aren't that, they're just for controlling visual placement of controls. The real MVP of dataflow manipulation programming for non-programmers is Excel.
Form designers are editing a data structure. It's not just visual placement of controls; it's the settings on those controls: content, appearance, behaviour, event bindings. And not just controls, also components.
VB and Delphi have components, non-visual controls that live on design surfaces, where their properties can be edited. Things like database connections and table connectors, which can further be linked up with data aware controls. Whole simple form-based CRUD applications can be built without any code at all, merely by wiring together components.
The wiring is data.
Some of these interfaces might be really fun to implement, but there are very few of them that I think I'd want to use.
For me, writing code is about the easiest thing I do. Other than identifier completion, I don't need much beyond a basic text editor.
On the other hand, understanding code -- not just exploring it, but understanding it -- is the the hardest thing I have to do, and that's where a good UI could be helpful. I only made a superficial scan, but it didn't really seem like any of these UIs were designed to do that.
I don't really need a map to a code base so much as I need a tool for making my own maps (not just one!) into the codebase as I try to understand it. Or understand at least enough of it to do my current task.
Phenomenal compendium, I love it. There always have been a lot of visual approaches to programming, but only a few had a relative success, and always restricted to a very limited domain. There is no competition to raw text editing when dealing with big, general problems and projects (yet). Why? Information density, convenience and fast editing may be the main factors, imho..
I think a more likely reason is keyboards. The main input device we use to interact with computers is a typewriter descendant, optimized for typing text. Mice are flexible but keyboards allow for more complex input.
Touch devices (like touchscreens or touchpads) could have helped with more direct manipulation of visual elements but they are always used on very limited (in terms of functionality and form factor) systems. On desktop computers they are far from ubiquitous and 99% of the time they just act as subpar mice.
I've come to think that one of the other problems non-textual approaches need to solve to win at scale is that they need to somehow get rid of their coordinates. Right now, most of the non-text oriented approaches I see mean that I have to not only understand the same thing I'd have to understand in a textual representation, but I also have the additional complexity of all of my code having coordinates in a 2D space, and then if you start drawing lines on the screen for relationships, additional accidental complexity in the state of the lines too.
The more I think about it, the more I think the people who tend to make criticisms about textual representations and then argue in favor of visual representations are not just wrong, but usually have things actively backwards; almost every criticism they make about text is actually a positive aspect of it. Not having a literal line appear from a call site to what it is calling, for instance, is a net positive, not a problem. I think this is why visual stuff has been bubbling slowly for decades and just never seems to get anywhere the way its advocates hope.
This must be a big part of it. Another is the kinds of computing environments that the majority of users, developers, and even CS researchers live and work inside of. For there to be truly new UIs the question of "what is symbolic representation in the medium of computing" (that is beyond written text) needs to be taken seriously, and that's just not going to happen when everything is Unix
It's not that clear cut. Large portion of programming activity already involves keyboard shortcuts etc rather than straight out typing code. I'd expect any good non-text based code editor to still be fully keyboard driven, or at least as much as current-day editors
Maybe the solution is that we need to get away from general purpose languages for most software. There could be very powerful visual languages that one could develop given a specific domain. I work in pharma and I see a ton of scope for this, especially for the automation of daily work. Scientists might not be the best with C but they could be amazing given a DSL in a domain that they completely grok. I've begun working more and more with Siemens' S7 GRAPH language for sequential process control and its bloody amazing how it lets me visually represent complicated state transitions, loops and conditional logic in a very easy to understand intuitive way. It also lets you establish pre/post invariants, etc. I used to scoff at PLCs thinking I could trivially write than in C, but even ladder logic has humbled me more than a few times.
I think one reason is self-selection bias: Because all programming languages are text, everyone who is a programmer is necessarily adept, comfortable, and intimately familiar with representing programs as text.
There is a whole other kind of programmer which doesn't exist yet— a visual coder— all of whom are doing something else right now, because there are no tools for them. It takes a large leap of imagination for a text-coder to empathize with these still non-existent visual-coders and make the tool that they need, and a large leap of imagination to envision a complete visual coding system from the ground up which can be used to solve actual problems and compete with the very mature text code tools that are already out there.
In short, "massive amounts of cultural inertia".
Previous discussion: https://news.ycombinator.com/item?id=14290909
Copying erikj's comment from over there:
-----------------
I think Symbolics' Dynamic Windows and CLIM could be added to this gallery as well, as early examples of presentation-based user interfaces:
https://dspace.mit.edu/bitstream/handle/1721.1/41161/AI_WP_2....
ftp://publications.ai.mit.edu/ai-publications/2004/AIM-2004-005.pdf
I actually have an old Genera manual, and added some figures from it: https://docs.google.com/presentation/d/1MD-CgzODFWzdpnYXr8bE...
Funny that some like Mathematica and LabVIEW have not changed much for more than 30 years.
Those look mostly identical today.
The Mathematica screenshot can't be from 1988, it seems to be running on ubuntu 10 or newer judging from the title bar buttons look and position.
Edit: This is from 88 https://twitter.com/WolframResearch/status/10099016596554915...
I also don't think that LabView screenshot is from 1986. Looks like version 7-ish or 8-ish to me (mid to late 2000s).
Look here [0] if interested in the pioneers. Example [1].
[0] https://en.wikipedia.org/wiki/History_of_the_graphical_user_...
[1] https://en.wikipedia.org/wiki/Blit_(computer_terminal)#/medi...
No Visual Studio, .NET WinForms or WPF/XAML/Blend?
These are some of the most popular programmer interfaces of all time...
Is it only my bad memory or some of the dates are wrong:
Tinker 1981, showing a color Macintosh interface.
ProGraph 1983, also using an interface using colors.
Pecan 1984, the year the first Macintosh was released.
I think they just picked random screenshots. For example they show Visual Basic with a Windows 95 interface but it predates it by several years.
Yeah, the dates are an approximation of when it was developed, but many screenshots come from much later, especially for pre-web tools.
The Flash MX screenshot is actually of a newer version; MX didn't have ActionScript 3.0.
Fantastic collection. Would be lovely to have it in chronological order :)
Perhaps mention the link is to a 35mb PDF?