starboard/nplb: Fold nplb_evergreen_compat_tests into nplb proper

[yell0wd0g]

This is a follow-up to #4831, rolling nplb_evergreen_compat_tests into nplb proper.

Run via ./out/linux-x64x11_devel/nplb --gtest_filter=*CrashpadConfigTest*:*SabiTest*:*ExecutableMemoryTest*:*MaxFileNameTest*:*StorageTest*:*LoaderAppMetricsTest:*FontsTest* --

Two sets of test failed on my Linux and needed attention:

• CrashpadConfigTest.VerifyUploadCert was IIUC broken due to an
  extra ...app/cobalt/content... path manually added to the cert
  location. Looking at the GN ssl:copy_ssl_certificates target, it
  seems to agree with GetCACertificatesPath() in that the extra path
  is not needed. Discussing this with @y4vor, we decided to add a new
  helper target copy_ssl_certificates_evergreen to put the certs
  in the right spot.
• FontsTest.* needed deps on fonts:copy_font_data GN targets
  (both data_deps and deps). This worked, but I had to comment
  out the install_content = true statements in that BUILD.gn (it's
  unclear whether this would break other things). Said GN target
  uses exec_script and needs an exception in //.gn, this CL adds
  that too.

Asides from it, some unnecessary test classes are removed, and some
preprocessor logic simplified.

Bug: b/384819454

[andrewsavage1]

I think this is reasonable and LGTM but I'll leave it to Holden who knows more about these tests

[hlwarriner]

These tests are tricky and I think it unfortunately would be more trouble than it's worth to move them into nplb at this time.

They were never designed to run in CI and there are several reasons why they can't currently be expected to pass in CI across our reference devices: https://b.corp.google.com/issues/292138589#comment2. We could add filters for the problematic tests, but we'd probably end up filtering over half of them.

We may end up simply removing these tests soon. They were originally intended as a quick way for partners to assess progress in their Evergreen integrations, and their test coverage is very minimal compared to the Evergreen YTS tests. I need to revisit b/292138589 and make a proposal, but I think what we'll end up doing is retiring these tests and adding any extra coverage they provide to YTS.

Also, if we we do want to land this PR then we'll need to update a few partner docs (e.g., https://github.com/youtube/cobalt/blob/main/starboard/doc/evergreen/cobalt_evergreen_overview.md, https://developers.google.com/youtube/cobalt/docs/development/setup-android).

@yuying-y @lamdbui in case you disagree with anything here.

[yell0wd0g]

We may end up simply removing these tests soon.

I like this part. If these tests are nplb, then they should run there. Otherwise it's hard to justify their existence; the point of nplb is to be a single integration test.

[yell0wd0g]

@hlwarriner and I talked about this. Next PR proposal is:

• Build all of these tests when sb_is_evergreen_compatible (this CL asserts for it in the files instead of using it to sandwich the contents) as part of nplb.
• Some of the actual tests fail on e.g. Linux; this is because sb_is_evergreen_compatible doesn't mean sb_is_evergreen -- so this CL adds GTEST_SKIP() in those cases. (These test IIUC should also be skipped on e.g. Raspi, but not on e.g. RDK which is a fully fledged EG board).

@y4vor FYI

[hlwarriner]

@hlwarriner and I talked about this. Next PR proposal is:

• Build all of these tests when sb_is_evergreen_compatible (this CL asserts for it in the files instead of using it to sandwich the contents) as part of nplb.
• Some of the actual tests fail on e.g. Linux; this is because sb_is_evergreen_compatible doesn't mean sb_is_evergreen -- so this CL adds GTEST_SKIP() in those cases. (These test IIUC should also be skipped on e.g. Raspi, but not on e.g. RDK which is a fully fledged EG board).

@y4vor FYI

Just want to clarify something about sb_is_evergreen_compatible and sb_is_evergreen.

• sb_is_evergreen_compatible should only be set for "loader platforms" that can build the elf_loader and loader_app which, when run, can load the evergreen shared library (provided by a different platform). An evergreen partner building their partner-built components would want to set this in their port.
• sb_is_evergreen should only be set for the "evergreen platform" that is used to build the evergreen shared library. Partners should never set this, since these platforms are maintained by google (for a handful of architectures) and libcobalt.so is built by google.

What's kind of missing, and is highlighted by issue 2 from https://b.corp.google.com/issues/292138589#comment2, is something to capture whether an sb_is_evergreen_compatible platform is actively being used as an evergreen loader platform in the current build. In other words, linux-x64x11 is always evergreen-compatible but it isn't always used as an evergreen loader platform...sometimes it's just used to build a cobalt executable and to run plain, non-evergreen cobalt.

[yell0wd0g]

@hlwarriner and I talked about this. Next PR proposal is:

• Build all of these tests when sb_is_evergreen_compatible (this CL asserts for it in the files instead of using it to sandwich the contents) as part of nplb.
• Some of the actual tests fail on e.g. Linux; this is because sb_is_evergreen_compatible doesn't mean sb_is_evergreen -- so this CL adds GTEST_SKIP() in those cases. (These test IIUC should also be skipped on e.g. Raspi, but not on e.g. RDK which is a fully fledged EG board).

@y4vor FYI

Just want to clarify something about sb_is_evergreen_compatible and sb_is_evergreen.

• sb_is_evergreen_compatible should only be set for "loader platforms" that can build the elf_loader and loader_app which, when run, can load the evergreen shared library (provided by a different platform). An evergreen partner building their partner-built components would want to set this in their port.
• sb_is_evergreen should only be set for the "evergreen platform" that is used to build the evergreen shared library. Partners should never set this, since these platforms are maintained by google (for a handful of architectures) and libcobalt.so is built by google.

What's kind of missing, and is highlighted by issue 2 from b.corp.google.com/issues/292138589#comment2, is something to capture whether an sb_is_evergreen_compatible platform is actively being used as an evergreen loader platform in the current build. In other words, linux-x64x11 is always evergreen-compatible but it isn't always used as an evergreen loader platform...sometimes it's just used to build a cobalt executable and to run plain, non-evergreen cobalt.

To further the point of this CL, and perhaps erroneously, this is my understanding:

	sb_is_evergreen_compatible	sb_is_evergreen
Linux	yes	no
raspi	yes	No?
rdk	yes	yes!
aosp	yes	yes?
Android TV	No (!)	No
partner TVs (Samsung etc)	yes ofc	yes!

Ofc sb_is_evergreen -> sb_is_evergreen_compatible (with -> being a logical if-then, "implies").

[hlwarriner]

@hlwarriner and I talked about this. Next PR proposal is:

• Build all of these tests when sb_is_evergreen_compatible (this CL asserts for it in the files instead of using it to sandwich the contents) as part of nplb.
• Some of the actual tests fail on e.g. Linux; this is because sb_is_evergreen_compatible doesn't mean sb_is_evergreen -- so this CL adds GTEST_SKIP() in those cases. (These test IIUC should also be skipped on e.g. Raspi, but not on e.g. RDK which is a fully fledged EG board).

@y4vor FYI

Just want to clarify something about sb_is_evergreen_compatible and sb_is_evergreen.

• sb_is_evergreen_compatible should only be set for "loader platforms" that can build the elf_loader and loader_app which, when run, can load the evergreen shared library (provided by a different platform). An evergreen partner building their partner-built components would want to set this in their port.
• sb_is_evergreen should only be set for the "evergreen platform" that is used to build the evergreen shared library. Partners should never set this, since these platforms are maintained by google (for a handful of architectures) and libcobalt.so is built by google.

What's kind of missing, and is highlighted by issue 2 from b.corp.google.com/issues/292138589#comment2, is something to capture whether an sb_is_evergreen_compatible platform is actively being used as an evergreen loader platform in the current build. In other words, linux-x64x11 is always evergreen-compatible but it isn't always used as an evergreen loader platform...sometimes it's just used to build a cobalt executable and to run plain, non-evergreen cobalt.

To further the point of this CL, and perhaps erroneously, this is my understanding:

sb_is_evergreen_compatible sb_is_evergreen
Linux yes no
raspi yes No?
rdk yes yes!
aosp yes yes?
Android TV No (!) No
partner TVs (Samsung etc) yes ofc yes!
Ofc sb_is_evergreen -> sb_is_evergreen_compatible (with -> being a logical if-then, "implies").

No, I don't think that's quite right. sb_is_evergreen doesn't imply sb_is_evergreen_compatible, they are actually mutually exclusive.

A couple examples from 25.lts.1+ of evergreen-compatible platform and evergreen platform pairs, from our reference ports:

• Evergreen-compatible platform //starboard/linux/x64x11 can be used with Evergreen platform //starboard/evergreen/x64
• Evergreen-compatible platform //starboard/raspi/2 can be used with Evergreen platform //starboard/evergreen/arm/hardfp

[y4vor]

@hlwarriner and I talked about this. Next PR proposal is:

• Build all of these tests when sb_is_evergreen_compatible (this CL asserts for it in the files instead of using it to sandwich the contents) as part of nplb.
• Some of the actual tests fail on e.g. Linux; this is because sb_is_evergreen_compatible doesn't mean sb_is_evergreen -- so this CL adds GTEST_SKIP() in those cases. (These test IIUC should also be skipped on e.g. Raspi, but not on e.g. RDK which is a fully fledged EG board).

@y4vor FYI

Just want to clarify something about sb_is_evergreen_compatible and sb_is_evergreen.

• sb_is_evergreen_compatible should only be set for "loader platforms" that can build the elf_loader and loader_app which, when run, can load the evergreen shared library (provided by a different platform). An evergreen partner building their partner-built components would want to set this in their port.
• sb_is_evergreen should only be set for the "evergreen platform" that is used to build the evergreen shared library. Partners should never set this, since these platforms are maintained by google (for a handful of architectures) and libcobalt.so is built by google.

What's kind of missing, and is highlighted by issue 2 from b.corp.google.com/issues/292138589#comment2, is something to capture whether an sb_is_evergreen_compatible platform is actively being used as an evergreen loader platform in the current build. In other words, linux-x64x11 is always evergreen-compatible but it isn't always used as an evergreen loader platform...sometimes it's just used to build a cobalt executable and to run plain, non-evergreen cobalt.

To further the point of this CL, and perhaps erroneously, this is my understanding:
sb_is_evergreen_compatible sb_is_evergreen
Linux yes no
raspi yes No?
rdk yes yes!
aosp yes yes?
Android TV No (!) No
partner TVs (Samsung etc) yes ofc yes!
Ofc sb_is_evergreen -> sb_is_evergreen_compatible (with -> being a logical if-then, "implies").

No, I don't think that's quite right. sb_is_evergreen doesn't imply sb_is_evergreen_compatible, they are actually mutually exclusive.

A couple examples from 25.lts.1+ of evergreen-compatible platform and evergreen platform pairs, from our reference ports:

• Evergreen-compatible platform //starboard/linux/x64x11 can be used with Evergreen platform //starboard/evergreen/x64
• Evergreen-compatible platform //starboard/raspi/2 can be used with Evergreen platform //starboard/evergreen/arm/hardfp

The meaning of these flags is the following:
sb_is_evergreen: Applies only above Starboard and means it is an Evergreen/Hermetic build.
sb_is_evergreen_compatible: Applies only at the Starboard implementation and means the build would be used for Evergreen.

[yell0wd0g]

The meaning of these flags is the following:
sb_is_evergreen: Applies only above Starboard and means it is an Evergreen/Hermetic build.
sb_is_evergreen_compatible: Applies only at the Starboard implementation and means the build would be used for Evergreen.

Gotcha thanks for the explanation. I've changed the code to be more specific: SKIP those 3 test cases on platforms where we don't have the paths. Currently this is only Linux, since there it doesn't make sense to have to create e.g. kSbSystemPathFontDirectory just for passing an integration test. PTAL

[hlwarriner]

Left one comment requesting a small doc change but otherwise lgtm.

Let's just wait for approval from @y4vor , too.

[yell0wd0g]

@y4vor PTAL

@andrewsavage1 I had to do something fishy in starboard/content/fonts/BUILD.gn -- would you mind taking a look..? Thanks!