Effective prompt engineering requires understanding transformer architectures, attention mechanisms, and how these models actually process sequences—much like how deep software engineering requires understanding processors, memory hierarchies, and operating systems
That's not true. You don't understand "transformer architectures" etc (let's be honest, barely anyone of us does) and you don't expect your system/chat prompts to work with different models in the same architecture, or in different contexts of the same model. Prompt engineering gets a side eye simply because it's an ad-hoc iterative process of adjusting a prompt after a series of non-rigorous testing in a limited set of success and failure modes until it's just good enough.
In other words, without trying to sound smart, you make it work in a few minutes and hope that this will kinda work for every dialog. Which it doesn't do later, but now you're too lazy to create debug environments with similar contexts, so you blindly throw few more capsbold statements into the card.
Anyone who prompts a lot knows that there's almost nothing you can generalize around, except for specific prompt template syntaxes. And the knowledge how it works and practice tells you exactly that. You're navigating the space, elements of which and their combinations randomly overweight each other depending on incomprehensible factors. And creating a one-size-fits-all prompt is impossible and impractical in such conditions.
All that said, it's still non-trivial and can be a job title.
I don't disagree with the idea, but this:
Effective prompt engineering requires understanding transformer architectures, attention mechanisms, and how these models actually process sequences—much like how deep software engineering requires understanding processors, memory hierarchies, and operating systems
That's not true. You don't understand "transformer architectures" etc (let's be honest, barely anyone of us does) and you don't expect your system/chat prompts to work with different models in the same architecture, or in different contexts of the same model. Prompt engineering gets a side eye simply because it's an ad-hoc iterative process of adjusting a prompt after a series of non-rigorous testing in a limited set of success and failure modes until it's just good enough.
In other words, without trying to sound smart, you make it work in a few minutes and hope that this will kinda work for every dialog. Which it doesn't do later, but now you're too lazy to create debug environments with similar contexts, so you blindly throw few more capsbold statements into the card.
Anyone who prompts a lot knows that there's almost nothing you can generalize around, except for specific prompt template syntaxes. And the knowledge how it works and practice tells you exactly that. You're navigating the space, elements of which and their combinations randomly overweight each other depending on incomprehensible factors. And creating a one-size-fits-all prompt is impossible and impractical in such conditions.
All that said, it's still non-trivial and can be a job title.