“Simple” Level Control on an FPSO

One of the great things about the school I went to, The[i] University of Tulsa, is that the Chemical Engineering department was full of people with pretty extensive industry experience, and they solicited feedback from different employers on what coursework might add the most value for new graduates. During my Junior year, one of the things that came out of this feedback was that graduates with a better understanding of process control, which in general terms just means automated methods of controlling things like temperatures, pressure, flowrates, levels, or other variables in a system. In laymen’s terms, you know how you can never turn the red and blue knob or the fan speed setting in your car to make the temperature just the way you like it? They taught us how to be really awesome at stuff like that[ii].

On top of a dedicated process control course, the program began adding some additional process control elements to our unit operations lab class and elsewhere. I found the detailed material in this work, which included Matlab control simulations, control tuning methods, and even the use of Laplace transforms, fascinating. However, as is the case with most problems I have faced in the real world, most process control solutions employ almost none of the really interesting stuff you learn in college outside of the functions of a basic PID controller, which are explained pretty well on Wikipedia for those of you interested. You won’t need none of that to follow along with this story though, which covers one of the simplest and most costly control issues I have battled: How operating a facility on top of a rocking boat can absolutely wreck your ability to provide stable level control.

Full disclosure, this will be a less detailed version of an internal presentation I was selected to perform at my Chevron’s annual Facilities Engineering Virtual conference, so any of my colleagues interested in more technical details should tune into that sometime in October.

For those of you tuning out because you think process control is complicated or boring, wait for me to at least explain why the former isn’t true. The example I’ll [iii]discuss today will deal with basic level control in what is typically referred to as a knock-out (or KO) vessel, named this way because it “knocks out” any liquids entrained in the vapor phase before sending vapor downstream. Controlling the level in KO vessels is generally easy as they typically only get small amounts of liquid that just need to be periodically drained off, but are designed to be large enough to handle huge monstrous slugs of liquid that don’t happen that often. Imagine you have half a glass of water that is being refilled extremely slowly, and you want to maintain it at that 50% level regardless of how much is or isn’t being poured into the glass. If you have a straw, you might suck harder as water is poured in more quickly, or more slowly if just a trickle of water is being poured in. Or, if the flow is slow enough, you might just suck a little bit out every time the water reaches some level to keep the glass from overfilling. That’s a stupid example, but it gives you an idea of the type of dumb stuff engineers can get paid barrels of money to fix[iv].

Of course, we don’t suck fluid out of our process vessels with straws, we have drain pipes equipped with valves that open and close to allow fluid to flow out more or less quickly to maintain level. When the level falls below the desired amount, we close the valve and allow liquid level to build up. When the level goes high, we open the valve to bring it back down. We typically use the aforementioned PID controller to “tune” the response of this valve so it doesn’t open or close too quickly or slowly. To make this even easier, one of our “glasses” might be 12 feet high and four feet wide, and we can usually allow our levels readings to fluctuate quite a bit one way or another before there is a significant issue, so you wouldn’t think this is rocket science.

Part of the problem is that we almost never work with the entire glass. The range of “fullness” that is typically measure between 0 and 100% is limited to a small section of the vessel that has a level bridle attached. This is because you are severely restricted in how high or low you can really allow the level to go.

On the low side, you generally don’t want the level to fall so low that the vessel is sucked dry, which could allow any high pressure gas in the vessel to shoot out the drain line and overwhelm downstream equipment. There is often a significant safety margin applied so that this level does not fall below 30-40% of the measured level range. If it does, a safety valve on the outlet will typically close, refusing to allow any more liquid to escape.

On the high side, you have a different margin applied, usually one that doesn’t allow the level of the vessel to reach the point where fluid is filling the vessel, which is often already on the bottom half of the vessel. This is because the stream of incoming liquid and vapor needs a significant amount of height to allow liquid to separate from the vapor. If the liquid level of the vapor exceeds this inlet level, the incoming flow can blast this liquid upwards through the top of the vessel, not unlike what would happen in our example glass of water if instrad of sucking you decided to blow into the straw as hard as you could. Not only does this defeat the purpose of the “Knock-out” vessel, this gas is often going to sensitive equipment that is not equipped to handle liquids. If the knock out drum is upstream of a burner that isn’t designed to handle liquids, you may at best end up damaging equipment or at worst cause a rain shower of potentially burning liquid hydrocarbon to come raining out of a facility flare. Here’s what it looks like when a refinery flare meant to burn gas is sent liquid crude:

Why, yes, that is flaming oil falling to the ground. Not exactly an ideal situation.

Knock out drums are also typically upstream of compressors, which are machines that generate pressure to move fluid by compressing gases (as their name would indicate). The problem with sending liquids to a compressor in that liquids are not compressible, and can result in severe damage to compressors. Compressor are not only costly equipment to repair, but the loss of an important compression system can bottleneck or even shut down an entire plant for an extended period of time if there are not adequate spare compressors to maintain operation.

So suddenly our half-full glass starts to look half empty, as we realize our level bridle only measured about a sixth of the vessel height, and 50% really means half the distance between a third and halfway up the vessel. Our 12 feet of height has become 2 feet, and if we intend to keep our levels between 30 and 70% of that range to provide margin, that 2 feet becomes a range of 0.8 feet from high to low.

And now we get to the rolling.

I currently work aboard a Floating, Storage, Production, and Offloading Vessel, often referred to as an FPSO. When people ask me where I work, I say it’s a boat, because that’s what it looks like. In fact, the FPSO I work on is a converted oil tanker that had all of its main oil-production related equipment added on decades after the ship was built. One end of our ship, the turret, is fixed, allowing production lines to come into the ship without twisting and snapping. The rest of the ship swivels 360 degrees around the turret. I honestly don’t know enough to adequately explain the setup in words, but here’s an excellent animation that kind of gives an idea of how it works:

Neat-o! I still have no idea exactly how it works.

One problem with working on the ship like this is that when bad weather hits, or the wind and waves push perpendicular to each other, or the boat has just offloaded a million barrels of oil and has a high center of gravity, these factors can cause the vessel to roll pretty significantly. As some of you may have seen on my Facebook, I have measured the total combined roll of the FPSO on which I work as between 15-20 degrees total, meaning the ship can roll up to 10 degrees a single direction. The motion is slow enough (about 10-12 seconds per full roll) that it hasn’t made me seasick[v], but it can make sleeping nearly impossible at times, and persistent rolling throughout the course of a day does seem to make myself and others drowsy.

Using the tangent of 10 degrees as a guide (0.176), I can see that the level at the edge of this hypothetical vessel (diameter=4 feet, radius=2 feet), can increase or decrease by about 0.35 feet. However, the level bridle measuring the fluid level can be a foot outside of the vessel. If this is aligned in the same direction of the roll, the deviation in our theoretical vessel can be up to 0.53 feet each direction, or 1.06 feet total. In this case, even if we kept our vessel at exactly “50%” of the desired range when the vessel was stable, the indicated vessel level would fluctuate between 23.5 and 76.5%, easily enough to sound a high or low level alarm every 5 or 6 seconds as the vessel rolls.

Go back to that example of the glass of water with the straw and imagine that glass slowly being rocked a few degrees to each side. You would be able to see the water level was only rising on one side and falling an equal amount on the other and would be unlikely to completely lose your mind due to this, but individual level bridles only have the ability to see one side of our glass, and it drives them nuts. The automated controller will keep opening the drain valve and letting little blurps of fluid drain out every time the level exceeds the desired setpoint, until the level settles at a low enough level that the desired setpoint becomes the high level on this peak. This means the level alternates between “good” at the high point and “unacceptably low” on the other end, which means the low safety level usually gets violated and causes the backup safety valve on the drain to close, blocking in flow and allowing fluid to enter the vessel without draining until this safety valve is reset and re-opened, or the vessel reaches a high level and shuts down the plant before we get to that whole “rain of fire[vi]” scenario mentioned earlier.

As shown above, the level controller on the right has opened the control valve to the point where the “high” side of the level is right at the desired setpoint. However, the backup shutdown valve is generally tied to a separate, redundant level safety transmitter bridle in order to preserve multiple independent layers of protection against high and low level scenarios. Although it doesn’t really matter which side I drew this bridle on since the level will fluctuate on both sides, I have shown how the level safety transmitter currently reading “too low” on the “low” side, causing the shutdown valve to close. This is not a control valve, and once it closes it will remain closed until the low condition has been cleared, often requiring an additional manual input from the control room before being allowed to re-open.

Imagine being in a control room on a ship, already groggy and perhaps queasy from trying to focus intently on managing the plant while your entire world has been rocking for hours, and all the while potentially dozens of these alarms keep coming in every few seconds. Even worse, imagine knowing that each one of these level alarms means the system is on the verge of shutting itself down due to high or low level, resulting in potentially millions in lost revenue. Here’s another thought: Imagine programming the automatic PID controller, using these levels to automate the opening and closing of the drain valve on the vessel. Without any other information, the PID controller’s programming is to open the valve as is a flood of liquid is entering the vessel every few seconds. Good luck tuning this controller to maintain that average level at “exactly” 50%. The observed change in level during one 6 second period of roll can easily be as high as the change in level during “stable” operation of the vessel was allowed to fill up on its own for minutes or even hours depending on the service. The change in level doesn’t have to be the extreme case presented above either, a roll that results in level readings that fluctuate 10 or 20% are more than enough to ruin the function of a control loop not designed to handle them. In many cases this results in the automatic control being overridden altogether, forcing the control operator to make his best guess as to what the valve opening should be to maintain a relatively constant average level.

There are a few ways this can be addressed. A dampening factor can be added to each PID controller to reduce the effect of rolling, the valves of the level bridle can be partially closed as to mechanically “dampen” or reduce the rate at which the level bridle fills during periods of roll (I wouldn’t advocate for this), or the controller loop can use a time averaged value to eliminate control overreactions based on short periods of rolling. We chose to implement the third option, as this averaging could be put in one place and tracked through our distributed control system. It’s not a perfect solution, but averaging the last two or three values reported for level allowed the controller to take into account level readings at various points in the ships roll, and the high and low numbers typically canceled each other out. This enabled many of the controllers to remain engaged in automatic control mode, as their tuning parameters were able to be adjusted without having to worry as much about rolling behavior. I’d definitely recommend this approach to anyone else who sees their alarm panel go off like a Christmas lights every time their facility starts moving.

FPSO rolling is not a new phenomenon, and I’m sure I’m not breaking new ground, or…er, water. Anyone more familiar with the topic please let me know what you’ve seen in design or operation and let me know what I screwed up in the comments.

[i] Yes, the “T” is capitalized. I never really understood why.

[ii] Although taught only as an elective at The University of Tulsa, automobile A/C knob turning is actually a standalone major at most state schools in Oklahoma.

[iii] Remember what I said about being oversized? The vessel not only has to be tall enough to allow a big slug of liquid to come in, but it is also typically sized tall enough to have some non-trivial amount of residence time for liquid to flow into the vessel at that high rate. The vessels are also wide to allow gas to flow upwards slowly, making it less likely that high-velocity gasses will carry out any sizable droplets of water.

[iv] In defense of engineers, a buddy of mine who studied computer science at Tulsa made thousands of dollars on the side providing tech support, which from what I gather consisted almost entirely of going to people’s houses to unplug things and then plug them back in.

[v] Others on our ship haven’t been so lucky. And by others I mostly mean David.

[vi] Not to be confused with Reign of Fire. Obligatory shoutout to Deepak Chetty who shaved his head to look like Matthew McConaughey in that movie. Deepak’s Hard Reset is now available on Steam. I know that nobody ever clicks any of the links I put here because WordPress likes to remind me of this, but I encourage you to check this one out: http://store.steampowered.com/app/669970/Hard_Reset/