November’s T-SQL Tuesday is brought to us by Brad Schulz (blog). Brad’s topic is Prox ‘n’ Funx. In other words, procedures and functions. In the database. Yes, code in the database. This concept alone can bring out strong opinions. In a prior post (see item 3 of July’s T-SQL Tuesday Best Practices post) I discussed this briefly. In many situations I’ve had good results using stored procedures and functions, but they are a concept that can be abused, overused, etc. One such situation I encountered is what I’ll relay here today.
I was working with a firm that had scaling issues. The architecture was such that each of the firm’s clients had their own database, and each of these client databases had identical database objects. So when the firm got a new client, a new database for that client’s data was cloned from a template database that contained these identical database objects (e.g. empty tables plus views, stored procedures, functions, triggers, etc.).
Among the problems that were upsetting the CIO was the rate at which he was having to purchase new servers to host these client databases. By “new servers” I’m referring to 1U servers with two 20GB SCSI drives in a RAID1 array (this was many years ago). The CIO was looking at an Excel spreadsheet provided to him by system administrators showing space being used on these servers. The CIO said to me “Look at this, brand new clients, we don’t even have data loaded for them yet, but all of this space is already being consumed. That makes no sense!”
At which point I said “Well, it starts to make sense when you learn that each client’s database has over 27,000 stored procedures in it.” You can only imagine his shock and disbelief when learning this. But little by little, this had happened over time.
Everything imaginable in these client databases was done via stored procedures. Each table had a stored procedure for every type of function that could be done on the table. So every table had a store procedure each for INSERT, UPDATE, etc. as well as variations that provided extra features. Tables could only be modified through these stored procedures. Every table had a stored procedure for variations of queries. Views had one or more stored procedures for access. These stored procedures were then called by application code. Imagine ORM implemented through stored procedures. However, the data-access application code was also auto-generated via a solution that was tied to tables, so these stored procedures always seemed like an unnecessary layer of indirection to me. In any case, each table had a bunch of stored procedures tied to it for access and modification. So adding a table meant adding multiple new stored procedures. All queries and business logic were implemented in stored procedures, and each client database had every possible one of these stored procedures, even if that client would never need the functionality provided by them. In other words, if one client needed a stored procedure for some specific feature, all client databases contained that stored procedure. Some of these procedures were very large and unwieldy, so they would be broken up into smaller stored procedures that called one after another in sequence.
Rather than saying whether all this is good or bad, I’ll just say “Watch out.” This situation probably started off fine in the early days when there were two or three clients and a database had 25 tables or whatever. But with hundreds of identical databases, each containing hundreds of tables, this became an issue.
Stored procedures. Procedures that are stored, right? What is storing code (as opposed to data) going to mean for your solution over time? You need to think about such things when deciding that stored procedures are going to play a key role in your solution design.
Thanks to Brad Schulz for hosting T-SQL Tuesday #24, and thank you Adam Machanic (Blog|Twitter) for creating this monthly blog event and keeping it going for two years!