I use the inline Context filters a lot.
Surely something most users will do very often is to go through each Context to see how many tasks lie within each one in turn. However if feels like quite a lot of clicks to do so, because as things stand you need to click a Context that is already open for a second time in order to close it.
I suggest that when the user already has a Context being filtered for and the user clicks on the next Context, that by default, rather than adding that Context to the existing Contexts being filtered for, that any existing Context filters should be turned off.
And in the less used situation where a user wants to see multiple contexts filtered for at once, I suggest that this should be made possible by first holiding down Control (or Shift?).
Trying to help