When testing things, the point is to try and change as little as possible in between the iterations.
So, if I am looking at training schedules, for instance, I want there to be a consistency of approach. ‘Consistency of approach’ means that the same players are always used: I load the particular save I am using, tweak something in regard to the coaches or training schedules, save the game under a new name, and let Football Manager do its thing for a season. Then, when I take measurements of the results, I know that any differences in attributes are down to either the changed training schedule, or the number or type of coaches I have employed to oversee the training. (The only other thing it could be is random variance.)
This is why Football Manager is easier to work with than the real world, because the variables are much easier to constrain, and you can be more confident that the results you are seeing are a consequence of something you have changed. Chance plays its part, but you can eliminate at least some of that by repeating the same save, with the same coaches, and the same training schedule, numerous times, and thus establish tolerance bands: a training schedule which gives you an attribute lift of between +3 and +12 is a better one than a schedule which yields between -2 and +4.
That repetition is something I have only just started, and will continue to work with – but training schedules are not the only thing I am interested in.
It’s time to look at tactics, too.
I’ll eventually put up a table similar to the training schedules one, but the methodology is exactly the same. I have taken two clubs, edited their player attributes, and will now start playing matches between the two. The players are all exactly the same age – 23 – and have the same personality – balanced.
The team I am managing has an average attribute of about 10 (I say ‘about’ because of that pesky random variation, which means some attributes are 10, and some are 11) and the opponent has an average attribute of about 13 (which means some attributes will be 13, and others 14.) In other words, the opponent’s weakest points will be two points above my strongest ones, and there will be a gap of four attribute points between the opponent’s strongest attributes, and my worst ones.
And now I’ll play lots of games between the two teams, and see what does well at holding back the AI, and what does not. Drawing with the computer, or losing whilst having more shots than it, would appear to pinpoint a tactic which is worth persevering with, and a defeat is, well, a defeat.
Hopefully, the same blind, drunken stumbling will see me make some headway on the tactical side, as well as the training side.