Using the Proper Tool for the Job

calendar May 23, 2024 · Thoughts  ·
Share on: linkedin copy

I was in a meeting the other day where someone stated that "writing SQL isn't real engineering." I was taken aback by this, and initially didn't even know how to respond other than to say that sometimes SQL is the best tool to solve a problem, and other times it isn't. The blanket statement that one programming language is somehow better or more "real engineering" than another seemed incredibly narrow-minded. I've done some reflecting on this since, and wanted to share.

Background

Workshop tools on a pegboard
Before I start, I should point out that while I don't agree with the above statement at all, my career has spanned both sides of it. Longtime readers of this blog will know I built my career with SQL Server and later moved towards Azure SQL Database and other data offerings. I spent many years as a DBA, and continue to love both query tuning and understanding the engine internals that make databases work.

Several years ago I was given the opportunity to leverage my database skills in the direction of software engineering. These days I serve as a database expert on a software team that focuses on building tools and services which automate database maintenance and performance tuning at cloud scale. While my knowledge of the database is still very important, I now utilize it differently, and am usually writing C# or PowerShell on any given day. I can honestly say that being able to see a problem from the perspective of both a database administrator and a software engineer has been very eye-opening.

SQL is great at some things

There are many situations where SQL is the best - and sometimes only - solution to a problem.

  • If you need to query data from a relational database, some flavor of SQL is likely how you'll do that.
  • If you're trying to tune a query or investigate why it's performing poorly, looking at the SQL should be one of the early stages of troubleshooting. Even if it doesn't end up being the solution, understanding the workload is key to improving a process and may help guide any next steps.
  • If you're trying to create or configure a database, SQL is probably how you're doing this too, though likely with DDL statements.

If your environment contains relational databases, there are lots of reasons why SQL will be necessary. And while it's very easy to write a basic SELECT statement, writing good SQL is not easy. It took me years to really learn it and to understand all the little things that can make or break a query and its performance. Query tuning is as much an art as it is a skill, and improving at it comes only with experience. I'm still learning it in many ways, and I hope to never stop.

...and not great at others

For all the things that SQL is great at, there's plenty more things that it's not. SQL is a tool designed for a specific purpose - interacting with relational databases. When attempts are made to utilize SQL for other things, it may or may not work as well. The old adage of "having a hammer doesn't mean that every problem is a nail" comes to mind.

  • Control logic in SQL, while possible, isn't always a great experience. I'd much rather leverage a programming language such as C# or PowerShell if they are an option.
  • If you're trying to create something like an application or a service, setting up a scheduled SQL script probably isn't going to satisfy your needs.
  • Testing SQL code (or stored procedures or functions) is absolutely possible, but I've had a much better time testing applications written in other languages.
  • Need something to execute in multiple threads? SQL definitely is of no help to you here.

The list can go on and on, but my point is that SQL is really good at some things, and not as good at other things. And that's okay. Computer science has no silver bullet; many different tools exist for different purposes. Solving a problem with SQL, or any other language or combination thereof, is absolutely an engineering type of skill.

So what's the best language?

There is of course no answer to this question, and anyone trying to convince you that one language is better than another, or that one product is better than another because of the language it's implemented in, probably doesn't understand what they're talking about. Furthermore, they are making an extremely shortsighted statement.

MySQL is mostly written in C++. PostgreSQL is mostly written in C. I understand that SQL Server is mostly written in C++ as well. But I would never judge any of these products, or declare them to be "more" or "less" engineered based on the language they were implemented in.

What is "real engineering"?

Someone I look up to once told me that engineering is using tools to solve problems, but that the focus is never on the tools. Being an engineer means being able to utilize the tools and constraints you are given to solve those problems in the best manner possible.

I'll build on that a little more and say that part of being a good engineer is understanding the advantages and limitations of the tools you possess, enabling you to make informed decisions about how to leverage them. It's about finding compromise between tools, constraints, and often people - in order to build effective solutions.

If SQL is one of the tools you are given and you are building effective solutions with it, then congratulations, you are doing real engineering. Don't let anyone try to convince you otherwise.