idb/XCTestBootstrap
Tien-Che Tsai c9a22ab761 Write Test Configuration to the container of a test bundle, instead of inside the test bundle
Summary:
This diff contains two changes related to the test configuration setup

 ---

The first one is the error checking of the instantiation of `testConfiguration` - it looks like a previous copy-n-paste? but by the context we want to test `testConfiguration`
https://www.internalfb.com/code/fbsource/[3a774059d7e3]/fbobjc/EndToEndTests/FBSimulatorControl/src/XCTestBootstrap/Configuration/FBTestRunnerConfiguration.m?lines=141-147%2C150%2C163-168

 ---

The second one is to write the `.xctestconfiguration` to where we persistent the `.xctest` bundle, instead of in the test bundle like the current implementation.
Rasons are

1) With D56303593, the `.xctest` would be a symlink to buck-out folders - which would be better not touching/modifying it to keep buck2's consistency.

2) With Remote Execution, the `buck-out` can be mounted in a different location then the `idb-mac-cwd`. While we write the file with `NSDataWritingAtomic`, this becomes an issue.
https://www.internalfb.com/code/fbsource/[3a774059d7e3f4411c4ee3e24ed2e6cb8a124130]/fbobjc/EndToEndTests/FBSimulatorControl/src/XCTestBootstrap/Configuration/FBTestConfiguration.m?lines=53

`-[NSData writeToFile:options:error:]` would return an error like this
```
2024-04-23T07:58:45-0700 error idb_xctest : command_name=run-macos-app event_end_time=1713884325473 event_start_time=1713884324886 event_type=FAILURE time=1713884325 [IDBUtilities] Location: /paragon/pods/247200188/home/execution/3/4e6c01d22bec45948119a6935472aa2f/work/fbobjc/Tools/idb-cli/Sources/IDBXCTest/MacOS/Run/RunMacOSAppCommandImpl.swift:94:28 method: run() - Error: Error Domain=com.facebook.XCTestBootstrap Code=0 "Failed to prepare test configuration" UserInfo={NSLocalizedDescription=Failed to prepare test configuration, NSUnderlyingError=0x600001046220 {Error Domain=NSCocoaErrorDomain Code=512 "The file “TpxMacosUnitAppTestsMac-40B19D7B-DECB-406C-B5DB-9798056B2D13.xctestconfiguration” couldn’t be saved in the folder “TpxMacosUnitAppTestsMac.xctest”.
Caused By: The operation couldn’t be completed. Cross-device link" UserInfo={NSFilePath=/private/var/folders/wz/23tx9kv14z96n2jxxnrsmjkw000xbj/T/idb-mac-cwd/1E0140F2-FF11-4739-9149-0DF54E8E9993-4816-0000EA2AC41A1EC6/idb-test-bundles/com.facebook.TpxMacosUnitAppTestsMac/TpxMacosUnitAppTestsMac.xctest/TpxMacosUnitAppTestsMac-40B19D7B-DECB-406C-B5DB-9798056B2D13.xctestconfiguration, NSLocalizedDescription=The file “TpxMacosUnitAppTestsMac-40B19D7B-DECB-406C-B5DB-9798056B2D13.xctestconfiguration” couldn’t be saved in the folder “TpxMacosUnitAppTestsMac.xctest”.
Caused By: The operation couldn’t be completed. Cross-device link, NSUnderlyingError=0x600001045ce0 {Error Domain=NSPOSIXErrorDomain Code=18 "Cross-device link"}}}}
Error: Location: /paragon/pods/247200188/home/execution/3/4e6c01d22bec45948119a6935472aa2f/work/fbobjc/Tools/idb-cli/Sources/IDBXCTest/MacOS/Run/RunMacOSAppCommandImpl.swift:94:28 method: run() - Error: Error Domain=com.facebook.XCTestBootstrap Code=0 "Failed to prepare test configuration" UserInfo={NSLocalizedDescription=Failed to prepare test configuration, NSUnderlyingError=0x600001046220 {Error Domain=NSCocoaErrorDomain Code=512 "The file “TpxMacosUnitAppTestsMac-40B19D7B-DECB-406C-B5DB-9798056B2D13.xctestconfiguration” couldn’t be saved in the folder “TpxMacosUnitAppTestsMac.xctest”.
Caused By: The operation couldn’t be completed. Cross-device link" UserInfo={NSFilePath=/private/var/folders/wz/23tx9kv14z96n2jxxnrsmjkw000xbj/T/idb-mac-cwd/1E0140F2-FF11-4739-9149-0DF54E8E9993-4816-0000EA2AC41A1EC6/idb-test-bundles/com.facebook.TpxMacosUnitAppTestsMac/TpxMacosUnitAppTestsMac.xctest/TpxMacosUnitAppTestsMac-40B19D7B-DECB-406C-B5DB-9798056B2D13.xctestconfiguration, NSLocalizedDescription=The file “TpxMacosUnitAppTestsMac-40B19D7B-DECB-406C-B5DB-9798056B2D13.xctestconfiguration” couldn’t be saved in the folder “TpxMacosUnitAppTestsMac.xctest”.
Caused By: The operation couldn’t be completed. Cross-device link, NSUnderlyingError=0x600001045ce0 {Error Domain=NSPOSIXErrorDomain Code=18 "Cross-device link"}}}}
```

The underlying POSIX 18 (`EXDEV`) is likely to be raised from the `rename` system call
https://www.man7.org/linux/man-pages/man2/rename.2.html
```
EXDEV  oldpath and newpath are not on the same mounted
              filesystem.  (Linux permits a filesystem to be mounted at
              multiple points, but rename() does not work across
              different mount points, even if the same filesystem is
              mounted on both.)
```
https://www.unix.com/man-page/FreeBSD/2/rename/
```
[EXDEV]		The link named by to and the file named by from are on different logical devices (file systems).  Note that this error
			code will not be returned if the implementation permits cross-device links.
```

We're not clear in Apple's actual implementation of `-[NSData writeToFile:options:error:]` with `NSDataWritingAtomic`, but it makes sense by the idiom of `NSDataWritingAtomic` - some renaming is involved.
Also for some other OSS version of the Foundation lib, we can see the `rename` system call is called.
- GNUStep
  - https://github.com/gnustep/libs-base/blob/master/Source/NSData.m#L2099
- apportable/foundation
  - https://github.com/apportable/Foundation/blob/master/System/Foundation/src/NSData.m#L646
  - https://github.com/apportable/Foundation/blob/master/System/Foundation/src/_NSFileIO.m#L232

It can be the symlink change in D56303593 make this path / device / mount point condition becomes a issue. By writing the `.xctestconfiguration` to where we store the `.xctest`, things are good now

Reviewed By: Nekitosss

Differential Revision: D56478995

fbshipit-source-id: a7ea532015ff1ea78e19ff9d2accd91df2d11497
2024-04-24 03:24:14 -07:00
..
Configuration Write Test Configuration to the container of a test bundle, instead of inside the test bundle 2024-04-24 03:24:14 -07:00
MacStrategies Do not install app bundle into Application folder for macOS 2024-04-18 12:52:40 -07:00
Reporters Signal process under test termination uppon crashes 2024-01-16 02:08:59 -08:00
Strategies Implement continuous coverage for logic tests 2024-01-16 02:39:36 -08:00
TestManager Fixing Lost Test Case Did Start 2023-11-15 02:19:25 -08:00
Utility Fixed file creation on RI 2024-04-04 04:58:35 -07:00
README.md Docusaurus2 Migration (#582) 2020-03-13 11:16:13 -07:00
XCTestBootstrap-Info.plist Make all Bundle Identifiers have the same prefix 2017-06-12 07:51:43 -07:00
XCTestBootstrap.h Update copyright headers from Facebook to Meta 2021-12-29 18:51:28 -08:00
XCTestBootstrap.xcconfig Use Framework Versioning to prevent Runtime Duplicate Symbols 2016-07-08 02:49:27 -07:00

README.md

XCTestBootstrap

A macOS library for executing XCTest bundles.

Part of the broader idb project.

Usage

You can use XCTestBootstrap directly as a Framework, but you may want to look at using idb as it provides saner defaults.