You need to copy in your output that measures S21 into the goals. Then, you need to modify it with

value( ( s21 expression ), X )

where X is the frequency you care about. If you need more than one freq, you will have to construct a more elaboratge expression that is, say, the sum of S21 at several frequencies, and try to maximize that.

Unfortauntely, you will find that the simulation is incredibly slow, even for very small neworks. This is a basic problem with the poor integration of the optimizer into the tool. It is wrapped around the outside, so there is much overhead. Make sure you are running in "interactive mode". This makes a very big difference. However, it hogs the license until you shut down the tool.