It does because it lessens your overall team FS, but it will allow you to go for longer streaks, increasing your modifier. Since you only get two fights per full charge, losses have a big impact.
Also, combine this with Saint2h's advice of starting with your lower FS fighters. The bigger the...