iOS: Update MB and JSON configs + enable Goma

iOS: Update MB and JSON configs + enable Goma

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which
is now updated to have component=static_library everywhere for iOS.
With this CL, all configs gets a full GYP+GN environment.

When flipping bots over to GN, the following line will need to be added
in addition to changing mb_type:
"additional_compile_targets": [ "all" ],

Goma was also enabled for all builders to reduce compile time.

BUG=589510
NOTRY=True

Committed: https://crrev.com/1c814e7b72ddfb98b1719a1acdba9b10734d930b
Cr-Commit-Position: refs/heads/master@{#13775}

[kjellander_webrtc, 2016-08-11 08:35:20]

Description was changed from

==========
MB: Enable MB for iOS GYP builders.

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This should turn on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

BUG=589510
==========

to

==========
MB: Enable MB for iOS GYP builders.

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This should turn on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

BUG=589510
NOTRY=True
==========

[kjellander_webrtc, 2016-08-11 09:01:32]

Apparently it tries to run build/gyp_chromium instead of our own GYP script, I
will investigate how to make it run the right script.

[kjellander_webrtc, 2016-08-12 04:30:21]

Description was changed from

==========
MB: Enable MB for iOS GYP builders.

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This should turn on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

BUG=589510
NOTRY=True
==========

to

==========
MB: Enable for iOS GYP builders + fix static config

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This should turn on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

BUG=589510
NOTRY=True
==========

[kjellander_webrtc, 2016-08-12 04:30:54]

Description was changed from

==========
MB: Enable for iOS GYP builders + fix static config

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This should turn on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

BUG=589510
NOTRY=True
==========

to

==========
MB: Enable for iOS GYP builders + fix static config

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This should turn on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

Debug builders were also using component=static_library by default,
so ensure all configs contain that setting.

BUG=589510
NOTRY=True
==========

[kjellander_webrtc, 2016-08-15 16:38:12]

kjellander@webrtc.org changed reviewers:
+ ehmaldonado@webrtc.org

[kjellander_webrtc, 2016-08-15 16:40:28]

This should be good to go in now that
https://codereview.chromium.org/2237793002/ is submitted.

[kjellander_webrtc, 2016-08-15 17:23:05]

euhm, I'm confused. It seems mb_config.pyl is used after all. I'll see what
happens if I update that instead then.

[kjellander_webrtc, 2016-08-15 18:37:58]

Description was changed from

==========
MB: Enable for iOS GYP builders + fix static config

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This should turn on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

Debug builders were also using component=static_library by default,
so ensure all configs contain that setting.

BUG=589510
NOTRY=True
==========

to

==========
Update iOS MB and JSON configs

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which is now
updated to have component=static_library everywhere for iOS.

BUG=589510
NOTRY=True
==========

[kjellander_webrtc, 2016-08-15 18:38:11]

Description was changed from

==========
Update iOS MB and JSON configs

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which is now
updated to have component=static_library everywhere for iOS.

BUG=589510
NOTRY=True
==========

to

==========
Update iOS MB and JSON configs

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which 
is now updated to have component=static_library everywhere for iOS.

BUG=589510
NOTRY=True
==========

[kjellander_webrtc, 2016-08-15 18:39:08]

Description was changed from

==========
Update iOS MB and JSON configs

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which 
is now updated to have component=static_library everywhere for iOS.

BUG=589510
NOTRY=True
==========

to

==========
iOS: Update MB and JSON configs + enable Goma

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which 
is now updated to have component=static_library everywhere for iOS.

Enabling Goma will speed up these builds significantly.

BUG=589510
NOTRY=True
==========

[kjellander_webrtc, 2016-08-15 18:39:27]

Patchset #6 (id:100001) has been deleted

[kjellander_webrtc, 2016-08-15 18:39:30]

Patchset #5 (id:80001) has been deleted

[kjellander_webrtc, 2016-08-15 18:39:34]

Patchset #4 (id:60001) has been deleted

[kjellander_webrtc, 2016-08-15 18:39:37]

Patchset #3 (id:40001) has been deleted

[kjellander_webrtc, 2016-08-15 18:39:41]

Patchset #2 (id:20001) has been deleted

[kjellander_webrtc, 2016-08-15 18:41:40]

Description was changed from

==========
iOS: Update MB and JSON configs + enable Goma

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which 
is now updated to have component=static_library everywhere for iOS.

Enabling Goma will speed up these builds significantly.

BUG=589510
NOTRY=True
==========

to

==========
iOS: Update MB and JSON configs + enable Goma

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which 
is now updated to have component=static_library everywhere for iOS.
With this CL, all configs gets a full GYP+GN environment.

When flipping bots over to GN, the following line will need to be added
in addition to changing mb_type:
"additional_compile_targets": [ "all" ],

Goma was also enabled for all builders to reduce compile time.

BUG=589510
NOTRY=True
==========

[kjellander_webrtc, 2016-08-15 18:46:18]

Patchset #1 (id:1) has been deleted

[kjellander_webrtc, 2016-08-15 18:55:37]

On 2016/08/15 17:23:05, kjellander_webrtc wrote:
> euhm, I'm confused. It seems mb_config.pyl is used after all. I'll see what
> happens if I update that instead then.

Whoa, I think I finally understood this. I repurposed the original CL a little
and added a few more comments in the JSON files to make it clear which configs
are used where.
Since all bots are now passing this is ready for review.

[kjellander_webrtc, 2016-08-15 19:00:44]

Complementary comments.

https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
File webrtc/build/ios/client.webrtc/iOS32_Debug.json (right):

https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
webrtc/build/ios/client.webrtc/iOS32_Debug.json:11: "GYP_DEFINES": [
The ios recipe module supports both dict and list as GYP_DEFINES, but I
converted it to be sure I'm using the same as the Chromium JSON files.

https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
File webrtc/build/ios/client.webrtc/iOS64_Debug_(GN).json (left):

https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
webrtc/build/ios/client.webrtc/iOS64_Debug_(GN).json:15: "target_cpu=\"arm\"",
Spotted an error here, this was arm before - should have been arm64 (now
mb_config.pyl is what's actually being built, but this could have been
confusing).

https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/tryserv...
File webrtc/build/ios/tryserver.webrtc/ios64_gn_rel.json (right):

https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/tryserv...
webrtc/build/ios/tryserver.webrtc/ios64_gn_rel.json:30:
"additional_compile_targets": [ "all" ],
Without this one, it tries to build 'All', which is a target that only exists
for GYP, not GN. It's a bit silly and maybe something we should fix. For
Chromium it's called 'gn_all' instead, which is equally bad though IMO. Ideally
it would be transparent but I'm not sure that's even possible.

https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/mb_config.pyl
File webrtc/build/mb_config.pyl (left):

https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/mb_config.p...
webrtc/build/mb_config.pyl:403: 'mixins': ['debug', 'shared']
This doesn't work for the built simulator tests, it needs to be static.

[ehmaldonado_webrtc, 2016-08-15 19:07:51]

On 2016/08/15 19:00:44, kjellander_webrtc wrote:
> Complementary comments.
> 
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> File webrtc/build/ios/client.webrtc/iOS32_Debug.json (right):
> 
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> webrtc/build/ios/client.webrtc/iOS32_Debug.json:11: "GYP_DEFINES": [
> The ios recipe module supports both dict and list as GYP_DEFINES, but I
> converted it to be sure I'm using the same as the Chromium JSON files.
> 
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> File webrtc/build/ios/client.webrtc/iOS64_Debug_(GN).json (left):
> 
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> webrtc/build/ios/client.webrtc/iOS64_Debug_(GN).json:15: "target_cpu=\"arm\"",
> Spotted an error here, this was arm before - should have been arm64 (now
> mb_config.pyl is what's actually being built, but this could have been
> confusing).
> 
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/tryserv...
> File webrtc/build/ios/tryserver.webrtc/ios64_gn_rel.json (right):
> 
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/tryserv...
> webrtc/build/ios/tryserver.webrtc/ios64_gn_rel.json:30:
> "additional_compile_targets": [ "all" ],
> Without this one, it tries to build 'All', which is a target that only exists
> for GYP, not GN. It's a bit silly and maybe something we should fix. For
> Chromium it's called 'gn_all' instead, which is equally bad though IMO.
Ideally
> it would be transparent but I'm not sure that's even possible.
> 
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/mb_config.pyl
> File webrtc/build/mb_config.pyl (left):
> 
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/mb_config.p...
> webrtc/build/mb_config.pyl:403: 'mixins': ['debug', 'shared']
> This doesn't work for the built simulator tests, it needs to be static.

So, what's being used when?

[kjellander_webrtc, 2016-08-15 19:14:36]

On 2016/08/15 19:07:51, ehmaldonado_webrtc wrote:
> On 2016/08/15 19:00:44, kjellander_webrtc wrote:
> > Complementary comments.
> > 
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> > File webrtc/build/ios/client.webrtc/iOS32_Debug.json (right):
> > 
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> > webrtc/build/ios/client.webrtc/iOS32_Debug.json:11: "GYP_DEFINES": [
> > The ios recipe module supports both dict and list as GYP_DEFINES, but I
> > converted it to be sure I'm using the same as the Chromium JSON files.
> > 
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> > File webrtc/build/ios/client.webrtc/iOS64_Debug_(GN).json (left):
> > 
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> > webrtc/build/ios/client.webrtc/iOS64_Debug_(GN).json:15:
"target_cpu=\"arm\"",
> > Spotted an error here, this was arm before - should have been arm64 (now
> > mb_config.pyl is what's actually being built, but this could have been
> > confusing).
> > 
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/tryserv...
> > File webrtc/build/ios/tryserver.webrtc/ios64_gn_rel.json (right):
> > 
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/tryserv...
> > webrtc/build/ios/tryserver.webrtc/ios64_gn_rel.json:30:
> > "additional_compile_targets": [ "all" ],
> > Without this one, it tries to build 'All', which is a target that only
exists
> > for GYP, not GN. It's a bit silly and maybe something we should fix. For
> > Chromium it's called 'gn_all' instead, which is equally bad though IMO.
> Ideally
> > it would be transparent but I'm not sure that's even possible.
> > 
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/mb_config.pyl
> > File webrtc/build/mb_config.pyl (left):
> > 
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/mb_config.p...
> > webrtc/build/mb_config.pyl:403: 'mixins': ['debug', 'shared']
> > This doesn't work for the built simulator tests, it needs to be static.
> 
> So, what's being used when?

When the 'generate_build_files' step runs, which generates the project files
that are use to compile, the MB configs in webrtc/build/mb_config.pyl that
matches the name of the builder are being used.
The environment in the JSON files is used during runhooks and by the recipe to
decide if it needs to bring up Goma etc.
I don't like how it's setup, but it's how it's working in Chromium. The
alternative is to completely maintain our own iOS build and configs, and
currently I think this is still better.

[ehmaldonado_webrtc, 2016-08-16 09:32:13]

On 2016/08/15 19:14:36, kjellander_webrtc wrote:
> On 2016/08/15 19:07:51, ehmaldonado_webrtc wrote:
> > On 2016/08/15 19:00:44, kjellander_webrtc wrote:
> > > Complementary comments.
> > > 
> > >
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> > > File webrtc/build/ios/client.webrtc/iOS32_Debug.json (right):
> > > 
> > >
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> > > webrtc/build/ios/client.webrtc/iOS32_Debug.json:11: "GYP_DEFINES": [
> > > The ios recipe module supports both dict and list as GYP_DEFINES, but I
> > > converted it to be sure I'm using the same as the Chromium JSON files.
> > > 
> > >
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> > > File webrtc/build/ios/client.webrtc/iOS64_Debug_(GN).json (left):
> > > 
> > >
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/client....
> > > webrtc/build/ios/client.webrtc/iOS64_Debug_(GN).json:15:
> "target_cpu=\"arm\"",
> > > Spotted an error here, this was arm before - should have been arm64 (now
> > > mb_config.pyl is what's actually being built, but this could have been
> > > confusing).
> > > 
> > >
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/tryserv...
> > > File webrtc/build/ios/tryserver.webrtc/ios64_gn_rel.json (right):
> > > 
> > >
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/ios/tryserv...
> > > webrtc/build/ios/tryserver.webrtc/ios64_gn_rel.json:30:
> > > "additional_compile_targets": [ "all" ],
> > > Without this one, it tries to build 'All', which is a target that only
> exists
> > > for GYP, not GN. It's a bit silly and maybe something we should fix. For
> > > Chromium it's called 'gn_all' instead, which is equally bad though IMO.
> > Ideally
> > > it would be transparent but I'm not sure that's even possible.
> > > 
> > >
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/mb_config.pyl
> > > File webrtc/build/mb_config.pyl (left):
> > > 
> > >
> >
>
https://codereview.webrtc.org/2239643002/diff/120001/webrtc/build/mb_config.p...
> > > webrtc/build/mb_config.pyl:403: 'mixins': ['debug', 'shared']
> > > This doesn't work for the built simulator tests, it needs to be static.
> > 
> > So, what's being used when?
> 
> When the 'generate_build_files' step runs, which generates the project files
> that are use to compile, the MB configs in webrtc/build/mb_config.pyl that
> matches the name of the builder are being used.
> The environment in the JSON files is used during runhooks and by the recipe to
> decide if it needs to bring up Goma etc.
> I don't like how it's setup, but it's how it's working in Chromium. The
> alternative is to completely maintain our own iOS build and configs, and
> currently I think this is still better.

lgtm

[kjellander_webrtc, 2016-08-16 09:32:44]

The CQ bit was checked by kjellander@webrtc.org

[commit-bot: I haz the power, 2016-08-16 09:32:48]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.webrtc.org/...

[commit-bot: I haz the power, 2016-08-16 09:42:04]

Description was changed from

==========
iOS: Update MB and JSON configs + enable Goma

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which 
is now updated to have component=static_library everywhere for iOS.
With this CL, all configs gets a full GYP+GN environment.

When flipping bots over to GN, the following line will need to be added
in addition to changing mb_type:
"additional_compile_targets": [ "all" ],

Goma was also enabled for all builders to reduce compile time.

BUG=589510
NOTRY=True
==========

to

==========
iOS: Update MB and JSON configs + enable Goma

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which 
is now updated to have component=static_library everywhere for iOS.
With this CL, all configs gets a full GYP+GN environment.

When flipping bots over to GN, the following line will need to be added
in addition to changing mb_type:
"additional_compile_targets": [ "all" ],

Goma was also enabled for all builders to reduce compile time.

BUG=589510
NOTRY=True
==========

[commit-bot: I haz the power, 2016-08-16 09:42:05]

Committed patchset #2 (id:140001)

[commit-bot: I haz the power, 2016-08-16 09:42:12]

Description was changed from

==========
iOS: Update MB and JSON configs + enable Goma

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which 
is now updated to have component=static_library everywhere for iOS.
With this CL, all configs gets a full GYP+GN environment.

When flipping bots over to GN, the following line will need to be added
in addition to changing mb_type:
"additional_compile_targets": [ "all" ],

Goma was also enabled for all builders to reduce compile time.

BUG=589510
NOTRY=True
==========

to

==========
iOS: Update MB and JSON configs + enable Goma

Turns out that if mb_type is missing in the JSON, GYP is run the
traditional way instead of having the MB configuration decide.
This turns on MB for those builders.
See https://codereview.chromium.org/2194703002 for how Chromium
switched from GYP->GN.

The JSON environment for GYP and GN is only used during runhooks
step since there are scripts that key on some of these environment variables.
The actual build that is compiled is defined by the MB config, which
is now updated to have component=static_library everywhere for iOS.
With this CL, all configs gets a full GYP+GN environment.

When flipping bots over to GN, the following line will need to be added
in addition to changing mb_type:
"additional_compile_targets": [ "all" ],

Goma was also enabled for all builders to reduce compile time.

BUG=589510
NOTRY=True

Committed: https://crrev.com/1c814e7b72ddfb98b1719a1acdba9b10734d930b
Cr-Commit-Position: refs/heads/master@{#13775}
==========

[commit-bot: I haz the power, 2016-08-16 09:42:14]

Patchset 2 (id:??) landed as
https://crrev.com/1c814e7b72ddfb98b1719a1acdba9b10734d930b
Cr-Commit-Position: refs/heads/master@{#13775}