Key management is nearly always the most difficult part of implementing any cryptosystem. As I mentioned in a previous post, encryption changes the problem of protecting a lot of secrets into the problem of protecting a single secret. That single secret is the key. If the bad guys can get your key, the game's over.
When approaching the key management problem for a database encryption system, you have to keep four requirements in mind: your system will need to store the key, use the key, back-up the key, and periodically change the key. Storage is exactly what you think. The key has to be written to a file or a database table. The trick, of course, is how to keep the key itself safe when it is stored.
Before we jump to protecting the key, let's consider where to put it. The first rule of key management is to store the key away from the data it is encrypting. This is the digital equivalent of not taping your house key to the front door. If you do store the key in the database and the database file is stolen, then you have no assurance that the attacker didn't pull the key out of the database file and use it to decrypt the data.
So your database is out. You could put it on a web or application server, but now a system administrator, or someone who has gained administrator privileges, can access the key. And since web and/or application servers generally have read access to the database, it would be trivial to compromise the data--this suggests an additional layer of protection gained by architecting your solution so that the web/app server does not need to read the encrypted data. Then by denying it read access to those columns or tables you severely limit the damage potential--but I digress.
Putting your key on the web or app server may offer the protection you need--check your threat model--but also keep in mind that these servers are often more exposed to attack. Overall, I don't recommend storing keys on the web or app servers. However, if such storage does fit your threat model, then you might also consider key splitting as an extra layer of protection.
Key splitting allows you to split the key into two or more parts and store the parts in different locations. To create a split key, generate two random numbers, A and B, and then let your key be K = XOR(A, B). You could store A on the app server and store B in the database. When the system starts up, it would read both A and B and XOR them together to obtain the key. This makes it more difficult to compromise the key, but in this case it mostly just slows an attacker down and should be seen more as a technique designed to keep the honest honest.
If key splitting isn't a great solution to protect a key stored on the web or application server, where should we store the key? The solution I like is to set up a dedicated machine to be a key vault. The key vault is a highly secured server dedicated to keeping keys secure and serving them only to other systems after a thorough authentication and authorization process. The key vault can be backed up separately and the backups can be stored separately. The administrators of the key vault should be folks who are not administrators of the business machines storing and processing the encrypted data.
In this architecture, the key vault becomes a business critical system: if it goes down, encryption and decryption stop. There are several techniques of reducing the risk here, but we'll leave those to another post (this post is already growing a tad long). There are also techniques such as using key-encryption keys to further protect the keys within the vault which I can go into later if there's interest.
At the beginning I mentioned four considerations for key management: storage, usage, backups, and rotation (changing the key). A key vault addresses storage and, to some extent, backups. We'll cover the remaining items in later posts.