First off, like many of the other so-called "laws" from the early days of modern chemistry / modern thermodynamics, Henry's Law is an approximate "law".

More importantly, the goal right now, and for at least the next couple of decades, is to merely reduce the rate at which carbon dioxide is being dumped into the atmosphere. The potential problems that might be associated with a lowering of carbon dioxide levels are problems for a future generation to solve.

Finally, it is true that the carbon dioxide dissolved in the oceans will start to come out of solution if carbon dioxide levels in the atmosphere actually do fall, and this will partially (but not completely) offset the reduction in human-generated carbon dioxide. That's a good thing rather than a bad thing. The growing dissolved carbon dioxide levels in the oceans is problematic in and of itself.