maxxxxx 5 years ago

Windows paths can be really infuriating. We constantly run into problems with the 255 file name length limit and even if you use the APIs to create longer paths then Explorer or other apps can't deal with these names.

It's hard to believe that after all these years this is still a problem but I guess a clunky foundation doesn't prevent one from being successful.

voldacar 5 years ago

The ugliness of the windows API never ceases to amaze me. Functions with names like RtlDetermineDosPathNameType_U and return types like PCWSTR just make my eyes bleed.

  • colatkinson 5 years ago

    My personal favorite is PCZZWSTR: a const pointer to a contiguous WCHAR array, where each substring is NUL-terminated, and the array itself is also terminated by a NUL. Thus (I assume) the ZZ--zero zero. I've only ever seen it used in the SHFILEOPSTRUCTW struct.

    • asveikau 5 years ago

      I feel like that construct, sometimes called "double-null terminated", is often inaccurately described.

      I prefer to think of it as a contiguous block of null terminated strings terminated by an empty string. (Just as '\0' terminates a C string, the empty string terminates such a list.)

    • alxmdev 5 years ago

      Multi-strings are also pretty common in the registry, see REG_MULTI_SZ.

    • voldacar 5 years ago

      >SHFILEOPSTRUCTW

      now that takes the cake

      • msclrhd 5 years ago

        How about SHITEMID in the shlwapi DLL? That's supposed to be SH-ITEM-ID.

      • whoopdedo 5 years ago

        Nah. That would be INITCOMMONCONTROLSEX. And the related LVM_SORTITEMSEX, DDSCAPSEX, and of course MPR_CREDENTIALSEX_0.

        • saagarjha 5 years ago

          With Microsoft's use of ALLCAPS and using an "Ex" prefix, this was all but guaranteed…

      • Barrin92 5 years ago

        if someone ever figures out the meaning of that phrase it's probably going to summon some cosmic horror that will be the end of us all

    • therein 5 years ago

      I've had some interesting experiences with SHFileOperationA on Wine. Star Citizen's updater wasn't working properly at one point due to it.

  • Const-me 5 years ago

    That's not an API. The article talks about some ntdll function, not supported by MS, and not documented by MS.

    The ugliness might be deliberate, to make developers conclude "that must be some Windows internal BS, probably undocumented" when they see it in debugger, profiler or other places.

    • asveikau 5 years ago

      > The ugliness might be deliberate, to make developers conclude "that must be some Windows internal BS, probably undocumented" when they see it in debugger, profiler or other places.

      Not at all. It is consistently named with internal NT APIs that are used like that intentionally. The people working at that layer follow such naming conventions without irony.

      • Const-me 5 years ago

        > consistently named with internal NT APIs

        I know, but these are mostly not part of WinAPI either. The public API surface has very good names, WriteFile or CreateDirectory is IMO better than POSIX write or mkdir.

        • rossy 5 years ago

          I don't know if they're good. WriteFile and CreateDirectory are fine, but when I think of the Windows API, I think of much longer function names. I had to use ConvertStringSecurityDescriptorToSecurityDescriptorA() once (and yeah, I had to use the ANSI version explicitly.) Another C/C++ developer might have called that function sec_sddl_to_desc(), which is much better IMO.

          • breakingcups 5 years ago

            I know I'm probably in the minority, but I vastly prefer the long-form name in your example. Modern IDE's (especially Visual Studio with its Intellisense) can be incredible with their autocomplete, superseding the need for short-form obtuse acronym-laden function names that you need to remember.

            So let's assume there's no time difference between writing the long or short form of that function. There's only one other direction to consider, which is reading.

            Try to approach the following question without personal bias. If you have never worked with either API, which function name would be more self-explanatory for you? ConvertStringSecurityDescriptorToSecurityDescriptorA or sec_sddl_to_desc?

          • asveikau 5 years ago

            One problem is that there is no one Windows style. The style guidelines, if you can call them that, depend on a few things like age and provenance of the component (some examples of pieces with distinct styles: filesystem, registry, GDI, NT API, COM). And sometimes you see a mix of these, or a component come along with its own variation. (SDDL and other security APIs is a good example of a unique style.)

            But sometimes people confuse unfamiliarity with ugliness, being bad or wrong, and having worked with a lot of these stuff I feel this is often a mischaracterization. Unfamiliarity doesn't always make it bad.

        • wvenable 5 years ago

          My favorite Unix function has to be "creat". It doesn't even make sense given the limitations of the systems at the time.

gumby 5 years ago

Shoutout for calling himself a "path'ological reverse engineer"

saagarjha 5 years ago

> we can only store at most 215-1 characters, which just happens to be 32,767

I think the formatting ate the superscript here.

  • hoseja 5 years ago

    <span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">2</span> <span style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">15</span>

    Yeah, I think it did.

yokohummer7 5 years ago

> Somewhere in Explorer something is passing one of these shell names to a file API, which is interpreting it as a Drive Relative path for the ‘:’ drive.

Isn't this just a pure bug? I'm quite surprised such an intimidating behavior still exists in today's Windows...

makomk 5 years ago

Gotta love how there's a whole chunk of complicated compatibility code to emulate the old DOS-ism of having per-drive current directories.

MarekKnapek 5 years ago

> Fortunately this doesn’t seem to work in the Explorer shell, but it does work when passed to native APIs such as CreateFile.

CreateFile is Win32 API, not Native API. NtCreateFile would be Native API.

Endy 5 years ago

And yet with all this, I still prefer Win16/Win32 paths and system layout as they are implemented for user needs compared to any UNIX/BSD/Linux pathing and system structure.

  • m45t3r 5 years ago

    Windows paths is full of WTFs:

    - The main driver is named C:\ because DOS used A:\ and B:\ for floppy disks! C'mon, floppy disks were not used as the main disk since god knows why (I think even MS-DOS itself was already supposed to run from HDDs and not floppy disks).

    - I can't have a file named LPT1, COM, AUX, PRN, NUL, etc. This would be fine, however I can't even have a file named aux.ext, for example. I know this is for compatibility purposes with DOS, however the fact that the error message is terrible does not help.

    -255 char limit. I know that you can use an API that allows more characters than this, however this does not work everywhere (last time I remember having problems to access those files in Windows Explorer; I think they fixed this in the latest versions of Windows 10, though).

    - Some common characters are prohibit in filenames, like ? or :. Unix has some limitations too, however the Windows are much more ridiculous.

    - In 64 bits systems, C:\Windows\SysWoW64\system32 is for 32 bits DLLs, while C:\Windows\system32 is for 64 bits DLLs. I really is curious why Microsoft choose this one, it is not like they needed for legacy software [1] (since 64 bits were new systems not old ones).

    I mean, most of the Windows WTFs are for compatibility purposes, however Unix has a much more sensible path rules even if you consider that Unix has much more history than Windows. There is much less WTFs in Unix paths (like /usr/bin vs /bin, or /opt vs. /usr/local).

    I can't really understand what is going on with Windows paths. Unix paths are far more sane to me IMO.

    [1]: If someone can link to me the reason why I would be thankful. I know that Microsoft engineers are not stupid, I just find it really curious why C:\Windows\system64 was not used, for example.

    • wvenable 5 years ago

      The short answer:

      "The 32 is just part of the name and doesn’t mean anything." -- Raymond Chen

      The long answer:

      "There are quite a number of existing 32-bit programs that hard-code the System32 path rather than calling the GetSystemDirectory function. When these programs are recompiled for 64-bit Windows, they will still try to access the System32 directory, expecting to find 64-bit files (because the program is now compiled 64-bit). Paths written into configuration files or the registry need to be meaningful to both 32-bit and 64-bit processes, yet need to refer to the appropriate directory depending on the “bitness” of the program doing the asking."

      https://docs.microsoft.com/en-us/previous-versions/technet-m...

      Basically System32 is the system directory regardless of the bitness of the OS. What matters is the bitness of the program being executed. If a 32bit program accesses System32 on a 64bit machine, it's silently redirected to a folder containing 32bit libraries.

      • ygra 5 years ago

        I think the only kind of program that need(ed) to know the distinction and access those paths directly and deliberately would be an installer for a different-fitness program. But that was also still in the days where programs would place their files in the Windows or system directory which hopefully no longer happens.

    • drfuchs 5 years ago

      To your first bullet point: For the first year and a half of its existence, the only shipping model of the IBM-PC came with one or two floppy drives, and no hard disk was available at all. And MS-DOS was the operating system of choice. https://en.m.wikipedia.org/wiki/IBM_Personal_Computer

    • legulere 5 years ago

      The Unix names are more ridiculous in my opinion: /usr/: in the beginning what is now /home/ but when the space in /bin/ got too tight binaries moved simply there

      • cyphar 5 years ago

        FreeBSD has a much better laid-out filesystem than Linux (which is what most people think of when they complain about "Unix directories").

    • wtallis 5 years ago

      I would guess that the system32 issue is they wanted you to be able to merely switch your compiler flags to 64-bit, without having to audit your entire codebase and build system for hard-coded references to system32. But whatever the reason, I'm sure they could have implemented something of equal or lesser complexity that would have also made sense.

      • JetSpiegel 5 years ago

        Just run 'grep -r'...

        Oh wait, Windows doesn't have grep.

    • twall 5 years ago

      [1] was also all about compatibility. It's quite amazing the lengths Microsoft went through to preserve compatibility during the 32 to 64 bit transition, and how well it continues to work to this day. That all came at the expense of confusing the hell out of mere users or developers looking back years later.

      It was important that programs compiled for a 32-bit windows could run without modifications on a 64-bit windows system. While some amount of 32-bit application recompiles or rewrites was inevitable, a surprising number of them could get by without modifications thanks to Microsoft's emulation & compatibility efforts.

      "wow64", or windows on window64, is an emulator of sorts to run 32 bit programs in a 64-bit windows environment with various behavior changes designed to trick 32-bit applications into thinking they were really running on a 32-bit system. The syswow64\system32 thing is one such effort.

      If I have a 32-bit program running I may have hard coded paths to files or libraries my program needs. If my program does a fopen("C:\Windows\System32\whatever", mode) expecting to find a dll and it's not there or cannot be opened because that's now where 64-bit dlls live, my program will blow up.

      So, when running in wow64 mode, the path "C:\Windows\System32" resolves to "C:\Windows\SysWow64\System32" when running a 32-bit program on 64-bit Windows. When running a 64-bit program, it resolves to what you'd "expect", "C\Windows\System32". That's a strange name for a 64-bit system folder, as you note, but imagine if it were something like system64 -- then if I want to support both 32 & 64 bit systems, I'd need different paths and logic in my program (and it's installer, and in all my tests, and ...) to switch between the two areas for 32-bit and 64-bit systems. By having the folder use the same name (from the program's perspective) everything gets a lot simpler.

      This same approach happens when you are looking at Program Files vs Program Files (x86), and in the registry, where HKLM\SOFTWARE\ is 64-bit, while HKLM\SOFTWARE\WOW6432Node\ is for 32-on-64.

      This emulation is so complete that it is actually difficult for a 32-bit program with awareness of Microsoft's emulation to actually find the files in SysWow64\System32. If for example, I had an installer program that wants to install my software for both kinds of system, it is difficult if you don't follow best practices and use 2 MSI installers for each bitness and wrap them in a bootstrapping installer program.

      I learned all this the hard way, by shipping a 32 & 64 bit windows library. We offer both 32 & 64 bit versions because it has to work with old 32-bit software, even to this day. It is quite confusing but if you squint hard, understand their reasoning and follow it through all the use cases it makes a decent amount of sense.

      • CamperBob2 5 years ago

        Of course, a lot of those problems come up because your installer is dropping stuff in system32. Stop doing that, please.

        • twall 5 years ago

          My problems were more with program files and the registry; I wouldn't dare go near system32. But the effects are all the same and they make sense holistically for Windows, given their long standing effort to preserve the compatibility of software written on their platform.

          That said the same library worked on a variety of Linux and Unix-like systems too, and packaging & installation process there was much more straightforward and predictable for me, the developer.

    • chris_wot 5 years ago

      You can't download and extract Unix make because of one of these reserved filenames (it's either CON or AUX, can't recall).