mesos-log/sprints/2017-01-13.md

Work log 2017-01-13

As always, the current-work dashboard is here.

Medium-term goals

As a reminder, the general goals for the early December/late January timeframe were agreed to focus almost entirely on stabilization:

1. Dramatically increase code coverage on the Windows agent. When we started, there were almost no tests enabled for the Windows build of the agent.
2. Onboard team members from Mike Neil's org.
3. Complete the CMake build system, deprecate the autotools build system.
4. Pave the way for adding agent authentication on Windows.

Current progress on these goals since early December

1. About 50% of our tests are now lit up in Apache master branch. By Monday, most of our critical path tests will be in review or checked in. Mesosphere has committed to lighting up all of this in the CI system this week and next week.
2. Andy is close to releasing his first important feature patch, replacing the prototype-quality WindowsLauncher with something that will work with the Mesos resource limit APIs.
3. CMake progress will start late next week (probably).
4. No progress on authentication, yet. This will be our first big new feature after stabilization.

Accomplished this week

The work this week continues to focus primarily on stabilization, specifically increasing test coverage on Windows code paths, making sure end-to-end tests work with ongoing changes to important infrastructure (particularly file descriptor patches):

Our main accomplishments are:

• DefaultExecutor and CommandExecutor tests are lit up and in review, with all patches needed for bugs they uncovered.
  • Tracked mostly in MESOS-6698 and MESOS-6870. This resolves MESOS-6839, which blocks a lot of our tests.
  • The last review in this chain is #53313. This review caps off a 20-review chain that fixes all the remaining issues.
  • Impact: This directly blocked many other tests, including the rest of the slave_tests.cpp, so this was extremely crucial.
• Added many critical path tests from MESOS-6695.
  • Particularly GC tests in #55328, and the health checker tests.
  • Still work to be done: a lot of the tests in this list are tests that won't be lit up until we port the fetcher and add authentication.
• Added Windows builds to the CI, but not Windows tests. This was entirely Mesosphere, but it's worth calling out anyway.
• Much of the work for a permanent version WindowsLauncher is in place, but none in review yet.
  • andschwa's first big feature.
  • Impact: This is the second-to-last feature we require for our alpha release. (The other being dpravat's Windows Containers.)

Focus for the upcoming week

• [dpravat] Make sure mpark's file descriptor changes work with the end-to-end tests. [CONSOLIDATED FROM LAST WEEK, since the build was broken by separate commits each day for most of this week.]
• [dpravat] [CONSOLIDATED FROM LAST WEEK] Work to approve get the long paths proposal approved by the core team this week.
  • (Daniel is working on this in his free time, so this may not happen :))
• [dpravat] [CONSOLIDATED FROM LAST WEEK] Resolve warning so we can effectively integrate with the Mesos CI. MESOS-5821.
  • (Daniel is working on this in his free time, so this may not happen :))
• [hausdorff] Work with Joseph to integrate the long backlog of patches that are up in my name.
• [hausdorff] Complete critical path tests.
• [hausdorff] Start working with dpravat to bring the Windows Container support in.
• [hausdorff] [CONSOLIDATED FROM LAST WEEK] Work with Srini (at IBM) to check in the Python and Java protobuf builds in CMake (depending on his and Joseph's availability).
  • Exit criteria: should be able to build Java and Python protobuf bindings on POSIX.
• [andschwa] Complete MESOS-6690, and continue onboarding.
• [andschwa] Integrate glog stack traces into Mesos crashes.