PySys

https://pysys-test.github.io/pysys-test/
Admin
PySys is a Python-based system testing framework that provides tools for creating, running, and...
Tokens:446,167
Snippets:3,681
Trust Score:9.7
Update:5 months ago
Show doc for...
Tokens:
Raw
### setup

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/writer/ci.html

Sets up the test environment for Travis CI. It configures logging, enables colored output, and starts a Travis CI fold section.

```APIDOC
## setup(**kwargs)

### Description
Configures the PySys test runner for a Travis CI environment. It sets default logging behavior (failures only), enables colored output compatible with Travis CI, and initiates a Travis CI fold for the test run.

### Method
`setup`

### Parameters
- `numTests` (int) - Optional - The total number of tests to run.
- `cycles` (int) - Optional - The number of times to cycle through the tests.
- `xargs` (any) - Optional - Additional arguments for the test runner.
- `threads` (int) - Optional - The number of threads to use for running tests.
- `testoutdir` (str) - Optional - The directory for test output.
- `runner` (object) - Optional - The test runner object.
- `**kwargs` - Arbitrary keyword arguments.

### Returns
None
```

--------------------------------

### start

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/baserunner.html

Starts the execution of a set of testcases. This method should not be overridden. Instead, override ``setup`` and/or call ``addCleanupFunction`` to customize runner behavior. It executes testcases multiple times based on the `self.cycle` attribute, saving output in cycle-specific directories.

```APIDOC
## start(self, printSummary=True)

### Description
Starts the execution of a set of testcases.

Do not override this method - instead, override ``setup`` and/or call ``addCleanupFunction`` to customize the behaviour of this runner.

The start method is the main method for executing the set of requested testcases. The set of testcases are executed a number of times determined by the C{self.cycle} attribute. When executing a testcase all output from the execution is saved in the testcase output subdirectory; should C{self.cycle} be set to more than 1, the output subdirectory is further split into cycle[n] directories to sandbox the output from each iteration.

:param printSummary: Ignored, exists only for compatibility reasons. To provide a custom summary printing implementation, specify a BaseSummaryResultsWriter subclass in the <writers> section of your project XML file.

:return: Use of this value is deprecated as of 1.3.0. This method returns a dictionary of testcase outcomes, and for compatibility reasons this will continue in the short term, but will be removed in a future release. Please ignore the return value of start() and use a custom BaseSummaryResultsWriter if you need to customize summarization of results.

### Parameters
#### Path Parameters
- **printSummary** (bool) - Ignored - Exists only for compatibility reasons.
```

--------------------------------

### start

Source: https://pysys-test.github.io/pysys-test/autodocgen/pysys.process.html

Starts the process.

```APIDOC
## start

### Description
Starts the process.
```

--------------------------------

### BaseRunner.setup

Source: https://pysys-test.github.io/pysys-test/autodocgen/pysys.baserunner.html

Setup method which may optionally be overridden to perform custom setup operations prior to execution of a set of testcases. All runner plugins will be setup and instantiated before this method is executed. Always ensure you call the super implementation of setup() before adding any custom logic, using `super().setup()`.

```APIDOC
## setup()

### Description
Setup method which may optionally be overridden to perform custom setup operations prior to execution of a set of testcases. All runner plugins will be setup and instantiated before this method is executed. Always ensure you call the super implementation of setup() before adding any custom logic, using `super().setup()`.

### Method
setup()
```

--------------------------------

### start()

Source: https://pysys-test.github.io/pysys-test/autodocgen/pysys.process.monitor.html

Starts the process monitoring in a background thread. This method performs initialization and then starts the thread.

```APIDOC
## start()

### Description
Starts the process monitoring in a background thread. This method performs initialization and then starts the thread.

### Returns
- **BaseProcessMonitor** - This instance.
```

--------------------------------

### start

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/process.html

Starts a process using the runtime parameters configured during instantiation. This method handles the initiation of both foreground and background processes.

```APIDOC
## start

### Description
Start a process using the runtime parameters set at instantiation.

### Raise
- pysys.exceptions.ProcessError: Raised if there is an error creating the process.
- pysys.exceptions.ProcessTimeout: Raised if the process timed out (foreground process only).
```

--------------------------------

### BaseRunner.start

Source: https://pysys-test.github.io/pysys-test/autodocgen/pysys.baserunner.html

Starts the execution of a set of testcases. Do not override this method - instead, override `setup` and/or call `addCleanupFunction` to customize the behaviour of this runner. The start method is the main method for executing the set of requested testcases. The set of testcases are executed a number of times determined by the `self.cycle` attribute. When executing a testcase all output from the execution is saved in the testcase output subdirectory; should `self.cycle` be set to more than 1, the output subdirectory is further split into cycle[n] directories to sandbox the output from each iteration. Use of this value is deprecated as of 1.3.0. This method returns a dictionary of testcase outcomes, and for compatibility reasons this will continue in the short term, but will be removed in a future release. Please ignore the return value of start() and use a custom BaseSummaryResultsWriter if you need to customize summarization of results.

```APIDOC
## start(printSummary =True)

### Description
Starts the execution of a set of testcases. Do not override this method - instead, override `setup` and/or call `addCleanupFunction` to customize the behaviour of this runner. The start method is the main method for executing the set of requested testcases. The set of testcases are executed a number of times determined by the `self.cycle` attribute. When executing a testcase all output from the execution is saved in the testcase output subdirectory; should `self.cycle` be set to more than 1, the output subdirectory is further split into cycle[n] directories to sandbox the output from each iteration.

### Parameters
#### Path Parameters
- **printSummary** – Ignored, exists only for compatibility reasons. To provide a custom summary printing implementation, specify a BaseSummaryResultsWriter subclass in the <writers> section of your project XML file.

### Returns
Use of this value is deprecated as of 1.3.0. This method returns a dictionary of testcase outcomes, and for compatibility reasons this will continue in the short term, but will be removed in a future release. Please ignore the return value of start() and use a custom BaseSummaryResultsWriter if you need to customize summarization of results.

### Method
start(printSummary =True)
```

--------------------------------

### BaseTest Setup Method

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/basetest.html

The setup method is intended for pre-execution actions and can be overridden by subclasses.

```APIDOC
## setup

### Description
Contains setup actions to be executed before the test is executed. This method may be overridden by individual test classes or custom BaseTest subclasses. If overridden, ensure to call the superclass's setup method.

### Behavior
If `setup` throws an exception, the `cleanup` method will still be called.
```

--------------------------------

### BaseTest Setup Method

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/basetest.html

Placeholder for setup actions before test execution. Subclasses can override this, ensuring to call the superclass's setup method. Exceptions during setup still trigger cleanup.

```python
	def setup(self):
		"""
		Contains setup actions to be executed before the test is executed. 
		
		The ``setup`` method may be overridden by individual test classes, or (more commonly) in a custom `BaseTest` 
		subclass that provides common functionality for multiple individual tests. However before implementing a custom 
		``BaseTest`` subclass with its own ``setup()`` method, consider whether the PySys concept of test plugins would meet 
		your needs. 
		
		If you do override this method, be sure to call ``super(BASETEST_CLASS_HERE, self).setup()`` to allow the 
		setup commands from the base test to run. 
		
		If ``setup`` throws an exception, the `cleanup` method will still be called, to clean up any resources 
		that were already allocated.
		
		"""
		pass		

```

--------------------------------

### setup

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/baserunner.html

Optional method to override for custom setup operations before a set of test cases are executed. Runner plugins are set up before this method is called.

```APIDOC
## setup()

### Description
Setup method which may optionally be overridden to perform custom setup operations prior to execution of a set of testcases.

All runner plugins will be setup and instantiated before this method is executed. 

Always ensure you call the super implementation of setup() before adding any custom logic, using
``super().setup()``.

### Method
setup
```

--------------------------------

### Python Coverage Writer Setup Example

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/writer/coverage.html

This code snippet demonstrates how to enable coverage for the PySys process itself, useful for testing PySys plugins. It requires pythonCoverageArgs to be set to a --rcfile argument.

```python
import coverage

args = self.getCoverageArgsList()
assert len(args)==1 and args[0].startswith('--rcfile='), 'includeCoverageFromPySysProcess can only be used if pythonCoverageArgs is set to "--rcfile=XXXX"'
mkdir(self.destDir)
cov = coverage.Coverage(config_file=args[0][args[0].find('=')+1:], data_file=self.destDir+'/.coverage.pysys_parent')
log.debug('Enabling Python coverage for this process: %s', cov)

# These lines avoid unhelpful warnings, and also match what coverage.process_startup() does
cov._warn_preimported_source = False
cov._warn_unimported_source = False
cov._warn_no_data = False
cov.start()
self.__selfCoverage = cov
```

--------------------------------

### Example Test Plugin Implementation

Source: https://pysys-test.github.io/pysys-test/ChangeLog.html

Demonstrates the structure of a custom test plugin, including setup, cleanup, and custom methods. Note that test plugins are no longer recommended as of PySys 2.2.

```python
class MyTestPlugin(object):
        myPluginProperty = 'default value'
        """
        Example of a plugin configuration property. The value for this plugin instance can be overridden using ``<property .../>``.
        Types such as boolean/list[str]/int/float will be automatically converted from string.
        """

        def setup(self, testObj):
                self.owner = self.testObj = testObj
                self.log = logging.getLogger('pysys.myorg.MyRunnerPlugin')
                self.log.info('Created MyTestPlugin instance with myPluginProperty=%s', self.myPluginProperty)

                testObj.addCleanupFunction(self.__myPluginCleanup)

        def __myPluginCleanup(self):
                self.log.info('Cleaning up MyTestPlugin instance')

        # An example of providing a method that can be accessed from each test
        def getPythonVersion(self):
                self.owner.startProcess(sys.executable, arguments=['--version'], stdouterr='MyTestPlugin.pythonVersion')
                return self.owner.waitForGrep('MyTestPlugin.pythonVersion.out', '(?P<output>.+)')['output'].strip()
```

--------------------------------

### Test Output Setup Method

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/writer/testoutput.html

The setup method for the Test Output writer. It validates required properties like destDir and fileIncludesRegex, normalizes paths, and prepares the destination directory for output collection.

```python
def setup(self, numTests=0, cycles=1, xargs=None, threads=0, testoutdir=u'', runner=None, **kwargs):
		for k in self.pluginProperties: 
			if not hasattr(type(self), k): raise UserError('Unknown property "%s" for %s'%(k, self))
		
		self.runner = runner
		if not self.destDir: raise Exception('Cannot set destDir to ""')
		if not self.fileIncludesRegex: raise Exception('fileIncludesRegex must be specified for %s'%type(self).__name__)

		self.destDir = os.path.normpath(os.path.join(runner.output+'/..', self.destDir))
		if pathexists(self.destDir+os.sep+'pysysproject.xml'): raise Exception('Cannot set destDir to testRootDir')
		
		# the code below assumes (for long path safe logic) this includes correct slashes (if any)
		self.outputPattern = self.outputPattern.replace('/',os.sep).replace('\\', os.sep)
		
		if self.destArchive: self.destArchive = os.path.join(self.destDir, self.destArchive)
		
		if os.path.exists(self.destDir):
			deletedir(self.destDir) # remove any existing archives (but not if this dir seems to have other stuff in it!)
		
```

--------------------------------

### IncludeLinesBetween Mapper Examples

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/mappers.html

Demonstrates various ways to use the IncludeLinesBetween mapper to control which lines are included based on start and stop conditions. These examples show usage with regex strings and lambda functions for matching.

```python
def _mapperUnitTest(mapper, input): return '|'.join(x for x in (applyMappers([line+'\n' for line in input.replace('<tab>', chr(9)).split('|')], [mapper])))
```

```python
_mapperUnitTest( IncludeLinesBetween('start.*', 'stopafter.*'), 'a|start line|b|c|stopafter line|d|start line2|e').replace('\n','')
```

```python
_mapperUnitTest( IncludeLinesBetween(startAt='start.*'), 'a|start line|b|c').replace('\n','')
```

```python
_mapperUnitTest( IncludeLinesBetween(startAfter='start.*'), 'a|start line|b|c').replace('\n','')
```

```python
_mapperUnitTest( IncludeLinesBetween(startAt=lambda l: l.startswith('start')), 'a|start line|b|c').replace('\n','')
```

```python
_mapperUnitTest( IncludeLinesBetween(stopAfter='stopafter.*'), 'a|stopafter|b|c').replace('\n','')
```

```python
_mapperUnitTest( IncludeLinesBetween(stopBefore='stopbefore.*'), 'a|b|stopbefore|c')
```

--------------------------------

### Start a Process

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/process.html

Starts a process using the runtime parameters defined during instantiation. Handles checks for working directory existence and runner abort status. For foreground processes, it starts the process and then waits for its completion.

```python
def start(self):
		"""Start a process using the runtime parameters set at instantiation.
		
		@raise pysys.exceptions.ProcessError: Raised if there is an error creating the process
		@raise pysys.exceptions.ProcessTimeout: Raised in the process timed out (foreground process only)
		
		"""
		self._outQueue = None # always reset
		
		if self.workingDir and not os.path.isdir(self.workingDir):
			raise Exception('Cannot start process %s as workingDir "%s" does not exist'% (self, self.workingDir))
		
		# unless we're performing some cleanup logic, don't permit new processes to begin after we've been told to shutdown
		if self.owner is not None and self.owner.isRunnerAborting is True and self.owner.isCleanupInProgress is False: raise KeyboardInterrupt()

		if self.state == FOREGROUND:
			self.startBackgroundProcess()
			self.wait(self.timeout)
		else:
			self.startBackgroundProcess()

```

--------------------------------

### Base Runner Setup Method

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/baserunner.html

Optional method to override for custom setup operations before test cases execute. Always call the super implementation first.

```python
def setup(self):
		"""Setup method which may optionally be overridden to perform custom setup operations prior to execution of a set of testcases.
		
		All runner plugins will be setup and instantiated before this method is executed. 
		
		Always ensure you call the super implementation of setup() before adding any custom logic, using
		``super().setup()``. 
		
		"""
		pass
```

--------------------------------

### Configure Test Object with Plugin Modes

Source: https://pysys-test.github.io/pysys-test/_sources/pysys/UserGuide.rst.txt

A test plugin can use the current test mode's parameters to configure the test object during setup. This example shows selecting a database helper based on 'database' parameter.

```python
class MyTestPlugin(object):
		def setup(self, testObj):
			# This is a convenient pattern for specifying the method or class 
			# constructor to call for each mode, and to get an exception if an 
			# invalid mode is specified
			dbHelperFactory = {
				'MockDatabase': MockDB,
				'MyDatabase2.0': lambda: self.startMyDatabase('2.0')
			}[testObj.mode.params['database']]
			...
			# Call the supplied method to start/configure the database
			testObj.db = dbHelperFactory()
```

--------------------------------

### Runner Plugin Setup Method

Source: https://pysys-test.github.io/pysys-test/autodocgen/pysys.baserunner.html

A minimal runner plugin requires a setup method. Use addCleanupFunction to schedule code execution after tests complete.

```python
def setup(self, runner):
        ...
        self.addCleanupFunction(...) # do this if you need to execute code after tests have completed
```

--------------------------------

### Test Output Writer Setup in PySys

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/baserunner.html

Sets up each test output writer, passing essential parameters like the total number of tests, cycles, and output directory. It includes error handling for setup failures.

```python
		for writer in list(self.writers):
			try: writer.setup(numTests=self.cycle * len(self.descriptors), cycles=self.cycle, xargs=self.xargs, threads=self.threads, 
				testoutdir=self.outsubdir, runner=self)
			except Exception:
				log.error("Caught %s setting up %s: %s", sys.exc_info()[0].__name__, writer, sys.exc_info()[1], exc_info=1)
				raise # better to fail obviously than to stagger on, but fail to record/update the expected output files, which user might not notice
```

--------------------------------

### Start Process with Custom Environment and Output

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/process/user.html

Starts a process, allowing customization of its environment, working directory, and standard output/error streams. It supports both foreground and background execution, with options for managing exit status and error handling.

```python
myexecutable = self.startProcess('path_to_my_executable',
				arguments=['myoperation', 'arg1','arg2'],
				environs=self.createEnvirons(addToLibPath=['my_ld_lib_path']), # if a customized environment is needed
				stdouterr=self.allocateUniqueStdOutErr('myoperation'), # for stdout/err files, pick a suitable logical name for what it's doing
				background=True # or remove for default behaviour of executing in foreground
```

--------------------------------

### Add testoutdir and runner to writer setup

Source: https://pysys-test.github.io/pysys-test/ChangeLog.html

The `setup()` method in `BaseResultsWriter` now accepts `testoutdir` to identify different test runs and `runner` to access runner configuration settings. This helps writers adapt to different testing contexts.

```python
testoutdir=, runner=
```

--------------------------------

### Start a Generic Process

Source: https://pysys-test.github.io/pysys-test/autodocgen/pysys.process.user.html

Starts a process with specified command, arguments, environment, and working directory. Supports foreground or background execution, custom timeouts, and stdout/stderr redirection. Use `allocateUniqueStdOutErr` for output file management.

```python
myexecutable = self.startProcess('path_to_my_executable',
        arguments=['myoperation', 'arg1','arg2'],
        environs=self.createEnvirons(addToLibPath=['my_ld_lib_path']), # if a customized environment is needed
        stdouterr=self.allocateUniqueStdOutErr('myoperation'), # for stdout/err files, pick a suitable logical name for what it's doing
        background=True # or remove for default behaviour of executing in foreground
        )
```

--------------------------------

### Install PySys from a wheel file

Source: https://pysys-test.github.io/pysys-test/Installation.html

If you have downloaded a PySys wheel (.whl) file, use this command to install it. Replace <VERSION> with the specific version number.

```bash
python -m pip install PySys-<VERSION>.whl

```

--------------------------------

### Start and Manage a Process

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/process/user.html

Starts a process with specified arguments, environment, and working directory. Handles foreground and background execution, exit status checking, and error reporting. It logs process execution details and manages process lifecycle.

```python
def startProcess(self, command, arguments=None, environs=None, workingDir=None, state=FOREGROUND, timeout=None, stdout=None, stderr=None, displayName=None, expectedExitStatus='==0', info=None, onError=None, abortOnError=True, ignoreExitStatus=False, quiet=False, processFactory=None):
		# in case stdout/err were given as non-absolute paths, make sure they go to the output dir not the cwd
		if stdout: stdout = os.path.join(self.output, stdout)
		if stderr: stderr = os.path.join(self.output, stderr)

		if not displayName: displayName = os.path.basename(command)
		
		if not environs: # a truly empty env isn't really usable, so populate it with a minimal default environment instead
			environs = self.getDefaultEnvirons(command=command)
		
		startTime = time.monotonic()
		
		# pass everything as a named parameter, which makes life easier for custom factory methods
		if processFactory is None: processFactory = pysys.process.helper.ProcessImpl
		process = processFactory(command=command, arguments=arguments, environs=environs, workingDir=workingDir, 
			state=state, timeout=timeout, stdout=stdout, stderr=stderr, 
			displayName=displayName, expectedExitStatus=expectedExitStatus, info=info, owner=self)
		
		def handleErrorAndGetOutcomeSuffix(process):
			if onError: 
				try:
					suffix = onError(process)
				except Exception as ex:
					self.log.debug('Ignoring exception from onError handler: ', exc_info=True)
				else:
					if suffix and isstring(suffix): return ' - '+suffix.strip()
			elif abortOnError:
				self.logFileContents(process.stderr, tail=True) or self.logFileContents(process.stdout, tail=True)
			return ''
			
		try:
			process.start()
			if state == FOREGROUND:
				correctExitStatus = pysys.utils.safeeval.safeEval('%d %s'%(process.exitStatus, expectedExitStatus), extraNamespace={'self':self})
				
				logmethod = log.info if correctExitStatus else log.warning
				if quiet: logmethod = log.debug
				logmethod("Executed %s, exit status %d%s", displayName, process.exitStatus,
					", duration %d secs" % (time.monotonic()-startTime) if (int(time.monotonic()-startTime)) > 10 else "")
				
				if not ignoreExitStatus and not correctExitStatus:
					if not stderr and not quiet: log.warning('Process %s has no stdouterr= specified; providing this parameter will allow PySys to capture the process output that shows why it failed', process)
					self.addOutcome(BLOCKED, 
						(
							('%s returned non-zero exit code %d'%(process, process.exitStatus)) 
							if expectedExitStatus=='==0' else
								('%s returned exit code %d (expected %s)'%(process, process.exitStatus, expectedExitStatus))
						)+handleErrorAndGetOutcomeSuffix(process), 
						abortOnError=abortOnError)
			elif state == BACKGROUND:
				(log.info if not quiet else log.debug)("Started %s with process id %d", displayName, process.pid)
			hasFailed = False
		except ProcessError as e:
			if not ignoreExitStatus:
				self.addOutcome(BLOCKED, 'Could not start %s process: %s'%(displayName, e), abortOnError=abortOnError)
			else: # this wouldn't happen during a polling-until-success use case so is always worth logging even in quiet mode
				log.info("%s", sys.exc_info()[1], exc_info=0)
		except ProcessTimeout:
			(log.warning if not quiet else log.debug)("Process %r timed out after %d seconds, stopping process", process, timeout, extra=BaseLogFormatter.tag(LOG_TIMEOUTS))
			process.stop()
			self.addOutcome(TIMEDOUT, '%s timed out after %d seconds%s'%(process, timeout, handleErrorAndGetOutcomeSuffix(process)), printReason=False, abortOnError=abortOnError)
		except BaseException:
			# if we don't do this then we can't cleanup foreground processes interrupted by serious failures like KeyboardInterrupt
			with self.lock:
				self.processList.append(process)
			raise
		else:
			with self.lock:
				self.processList.append(process)
				if displayName in self.processCount:
					self.processCount[displayName] = self.processCount[displayName] + 1
				else:
					self.processCount[displayName] = 1

		return process	
```

--------------------------------

### Start Manual Tester

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/basetest.html

Starts the manual tester UI, which guides a human through test implementation steps. Can run in FOREGROUND or BACKGROUND. Handles timeouts and automatic cleanup.

```python
if filedir is None: filedir = self.input

if not self.manualTester or self.manualTester.running() == 0:
```

--------------------------------

### PySys System Test Example

Source: https://pysys-test.github.io/pysys-test/GettingStarted.html

A comprehensive PySys system test demonstrating common methods for starting processes, file manipulation, and validation. This example tests a server application.

```python
__pysys_title__   = r""" MyServer startup - basic sanity test (+ demo of PySys basics) """

__pysys_purpose__ = r""" To demonstrate that MyServer can startup and response to basic requests.
  """

class PySysTest(pysys.basetest.BaseTest):
  def execute(self):
    # Ask PySys to allocate a free TCP port to start the server on (this allows running many tests in
    # parallel without clashes)
    serverPort = self.getNextAvailableTCPPort()

    # A common system testing task is pre-processing a file, for example to substitute in required
    # testing parameters
    self.copy(self.input+'/myserverconfig.json', self.output+'/', mappers=[
      lambda line: line.replace('@SERVER_PORT@', str(serverPort)),
    ])

    # Start the server application we're testing (as a background process)
    # self.project provides access to properties in pysysproject.xml, such as appHome which is the
    # location of the application we're testing
    server = self.startProcess(
      command   = self.project.appHome+'/my_server.%s'%('bat' if IS_WINDOWS else 'sh'),
      arguments = ['--configfile', self.output+'/myserverconfig.json', ],
      environs  = self.createEnvirons(addToExePath=os.path.dirname(PYTHON_EXE)),
      stdouterr = 'my_server', displayName = 'my_server<port %s>'%serverPort, background = True,
      )

    # Wait for the server to start by polling for a grep regular expression. The errorExpr/process
    # arguments ensure we abort with a really informative message if the server fails to start
    self.waitForGrep('my_server.out', 'Started MyServer .*on port .*', errorExpr=[' (ERROR|FATAL) '], process=server)

    # Run a test tool (in this case, written in Python) from this test's Input/ directory.
    self.startPython([self.input+'/httpget.py', f'http://localhost:{serverPort}/data/myfile.json'],
      stdouterr='httpget_myfile')

  def validate(self):
    # This method is called after execute() to perform validation of the results by checking the
    # contents of files in the test's output directory. Note that during test development you can
    # re-run validate() without waiting for a full execute() run using "pysys run --validateOnly".

    # It's good practice to check for unexpected errors and warnings so they don't go unnoticed
    self.assertGrep('my_server.out', ' (ERROR|FATAL|WARN) .*', contains=False)

    # Checking for exception stack traces is also a good idea; and joining them into a single line with a mapper will
    # give a more descriptive error if the test fails
    self.assertGrep('my_server.out', r'Traceback [(]most recent call last[)]', contains=False,
      mappers=[pysys.mappers.JoinLines.PythonTraceback()])

    self.assertThat('message == expected',
      message=pysys.utils.fileutils.loadJSON(self.output+'/httpget_myfile.out')['message'],
      expected="Hello world!",
      )

    self.logFileContents('my_server.out')

```

--------------------------------

### ProcessMonitor.start

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/process/monitor.html

Starts the process monitoring in a background thread. This method should be called on the main test thread to perform any necessary initial setup.

```APIDOC
## ProcessMonitor.start

### Description
Called on the main test thread to start monitoring in the background. Performs any required initialization of data structures then starts the background thread.

### Method
start()

### Returns
- This instance.

### Return Type
L{BaseProcessMonitor}
```

--------------------------------

### Start a process with custom arguments and environment

Source: https://pysys-test.github.io/pysys-test/autodocgen/pysys.baserunner.html

Use startProcess to launch a new process. Specify command, arguments, environment variables, working directory, and output/error streams. Background execution is supported.

```python
myexecutable = self.startProcess('path_to_my_executable', 
        arguments=['myoperation', 'arg1','arg2'],
        environs=self.createEnvirons(addToLibPath=['my_ld_lib_path']), # if a customized environment is needed
        stdouterr=self.allocateUniqueStdOutErr('myoperation'), # for stdout/err files, pick a suitable logical name for what it's doing
        background=True # or remove for default behaviour of executing in foreground
        )
```

--------------------------------

### Get default environment variables

Source: https://pysys-test.github.io/pysys-test/pysys/BaseTest.html

Creates a clean dictionary of environment variables for starting processes. Can be overridden to add common variables or customize behavior.

```python
self.getDefaultEnvirons()
```

--------------------------------

### Get Instance Count (Deprecated)

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/process/user.html

Deprecated method to return the number of processes started with a specific display name. Use `allocateUniqueStdOutErr` for recommended name allocation.

```python
def getInstanceCount(self, displayName):
        """(Deprecated) Return the number of processes started within the testcase matching the supplied displayName.

        :deprecated: The recommended way to allocate unique names is now L{allocateUniqueStdOutErr}

        The ProcessUser class maintains a reference count of processes started within the class instance 
        via the L{startProcess()} method. The reference count is maintained against a logical name for 
        the process, which is the C{displayName} used in the method call to L{startProcess()}, or the 
        basename of the command if no displayName was supplied. The method returns the number of 
        processes started with the supplied logical name, or 0 if no processes have been started. 
        """
```

--------------------------------

### Configure default environment properties in pysysproject.xml

Source: https://pysys-test.github.io/pysys-test/ChangeLog.html

Example of setting default environment properties for subprocesses started by BaseTest.startProcess. This allows customization of locale and temporary directory settings.

```xml
<property name="defaultEnvironsDefaultLang" value="en_US.UTF-8"/>

```

```xml
<property name="defaultEnvironsTempDir" value="self.output'"/>

```

--------------------------------

### Example usage of LIBRARY_PATH_ENV_VAR

Source: https://pysys-test.github.io/pysys-test/autodocgen/pysys.constants.html

Demonstrates how to construct environment variables for dynamic library paths, joining existing paths with a new one. Use os.pathsep for joining paths.

```python
environs={ LIBRARY_PATH_ENV_VAR: os.path.join([os.getenv(LIBRARY_PATH_ENV_VAR,''), mynewpath]) }
```

--------------------------------

### startManualTester

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/basetest.html

Starts the manual tester UI, which guides a human through test implementation steps. It can run in the foreground (blocking) or background (non-blocking). The UI can be terminated by timeout or explicitly via stopManualTester, and is automatically cleaned up by BaseTest.

```APIDOC
## startManualTester

### Description
Start the manual tester, which provides a UI to guide a human through the tests needed to implement this testcase. The manual tester user interface (UI) is used to describe a series of manual steps to be performed to execute and validate a test. Only a single instance of the UI can be running at any given time, and can be run either in the `FOREGROUND` (method will not return until the UI is closed or the timeout occurs) or in the `BACKGROUND` (method will return straight away so automated actions may be performed concurrently). Should the UI be terminated due to expiry of the timeout, a `TIMEDOUT` outcome will be added to the outcome list. The UI can be stopped via the `stopManualTester` method. An instance of the UI not explicitly stopped within a test will automatically be stopped via the `cleanup` method of the BaseTest.

### Method
`startManualTester(self, file, filedir=None, state=FOREGROUND, timeout=TIMEOUTS['ManualTester'])`

### Parameters
* **file** (string) - The name of the manual test xml input file
* **filedir** (string, optional) - The directory containing the manual test xml input file (defaults to the output subdirectory)
* **state** (enum, optional) - Start the manual tester either in the `FOREGROUND` or `BACKGROUND` (defaults to `FOREGROUND`)
* **timeout** (int, optional) - The timeout period after which to terminate a manual tester running in the `FOREGROUND`

```

--------------------------------

### setup

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/writer/api.html

Called before any tests begin. Before this method is called, for each property "PROP" specified for this writer in the project configuration file, the configured value will be assigned to `self.PROP`.

```APIDOC
def setup(self, numTests=0, cycles=1, xargs=None, threads=0, testoutdir=u'', runner=None, **kwargs):
    """ Called before any tests begin. 
    
    Before this method is called, for each property "PROP" specified for this 
    writer in the project configuration file, the configured value will be 
    assigned to `self.PROP`. 

    :param numTests: The total number of tests (cycles*testids) to be executed
    :param cycles: The number of cycles. 
    :param xargs: The runner's xargs
    :param threads: The number of threads used for running tests. 
    
    :param testoutdir: The output directory used for this test run 
        (equal to `runner.outsubdir`), an identifying string which often contains 
        the platform, or when there are multiple test runs on the same machine 
        may be used to distinguish between them. This is usually a relative path 
        but may be an absolute path. 
    
    :param runner: The runner instance that owns this writer. The default implementation of this methods sets 
        the ``self.runner`` attribute to this value. 
    
    :param kwargs: Additional keyword arguments may be added in a future release. 

    """
    if runner is not None: self.runner = runner
```

--------------------------------

### startProcess

Source: https://pysys-test.github.io/pysys-test/autodocgen/pysys.baserunner.html

Starts a new process with specified arguments and options. It allows for customization through a process factory and provides options for logging and error handling.

```APIDOC
## startProcess

### Description
Starts a new process with the specified arguments and options. This method is highly configurable, allowing for custom process implementations via `processFactory` and detailed control over logging and error handling.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
* **arguments** (_list_ _[str]_) – A list of strings representing the command and its arguments to be executed.
* **workingDir** (_str_) – The working directory for the new process. Defaults to the current working directory.
* **env** (_dict_ _[str, str]_) – A dictionary of environment variables for the new process. Defaults to inheriting the current environment.
* **quiet** (_bool_) – If True, suppresses INFO and WARN level logging for this process, only showing DEBUG level logs unless a failure occurs. Useful for repeated checks where success/failure are both valid outcomes.
* **info** (_dict_ _[str, obj]_) – A dictionary of user-defined information to be attached to the returned Process object. Useful for tracking process-specific metadata like ports or log file paths.
* **processFactory** (_callable_ _[**kwargs]_) – A callable (e.g., a class constructor) that returns an instance or subclass of `pysys.process.helper.ProcessImpl`. This allows for custom process behavior or modification of arguments/environment. The factory must accept `**kwargs` which will be populated with parameters from `pysys.process.Process`.
* **abortOnError** (_bool_) – If True, the test will abort if the process returns a non-zero exit status. Defaults to the project's `defaultAbortOnError` setting.

### Request Example
```python
# Example of using processFactory to modify arguments
def myProcessFactory(**kwargs):
    kwargs['arguments'] = kwargs['arguments'][0]+['my_extra_arg']+kwargs['arguments'][1:]
    return pysys.process.helper.ProcessImpl(**kwargs)

process = self.startProcess(
    arguments=['python', 'my_script.py'],
    processFactory=myProcessFactory,
    info={'port': 8080}
)
```

### Response
#### Success Response (200)
- **Process** (_pysys.process.Process_) – The process object representing the started process.
```

--------------------------------

### Exception Handling during Test Setup

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/baserunner.html

Catches exceptions during test setup and stores them for later reporting. This ensures that even if setup fails, the test runner can still report the error.

```python
except KeyboardInterrupt: # pragma: no cover
    self.runner.handleKbrdInt()
    raise
except Exception:
    exc_info.append(sys.exc_info())
```

--------------------------------

### Create a New PySys Project

Source: https://pysys-test.github.io/pysys-test/GettingStarted.html

Initialize a new PySys testing project by creating a directory and running the 'makeproject' command to generate the project configuration file.

```bash
> mkdir test
> cd test
> pysys.py makeproject

```

--------------------------------

### Pysystest XML Configuration Example

Source: https://pysys-test.github.io/pysys-test/pysys/TestDescriptors.html

Illustrates all possible configuration options for a test's pysystest.xml file.

```xml
<?xml version="1.0" encoding="utf-8"?>
<pysystest
      authors="userid1, userid2, Joe Bloggs" created="1999-12-31"
>

      <!-- Test titles should be concise but give a clear idea of what is in scope for this testcase.

      Good titles make it easy to find the test you need even when you have 100s/1000s of tests.
      Tests can be sorted by title so try to use common prefixes (e.g. ``Category: Title``) to group related tests
      together, and run ``pysys print -s=title`` to see how your title looks alongside the existing titles.

      Titles need to be human readable at-a-glance, so don't put ids (e.g. bug tracking numbers) in the title;
      the "purpose" or "requirements" are a better place for those details.

      The title can also be specified without XML as: __pysystest_title__ = """ ... """
      -->
      <title>My foobar tool - Argument parsing success and error cases</title>

      <!-- The purpose can also be specified without XML as: __pysystest_purpose__ = """ ... """
      -->
      <purpose><![CDATA[
              This is a good place for a fuller description of what is in and
              out of scope for this particular testcase.

      ]]></purpose>

      <!-- Comment/uncomment this to mark this test as skipped, which will stop it from executing. -->
      <skipped reason="This test is skipped until Bug-1234 is fixed"/>

      <!-- Specify the groups that this test will be tagged with, allowing them to be selected
      for inclusion/exclusion in test runs. Groups are usually named in camelCase.
      These groups are in addition to any from parent pysysdirconfigs if inherit=true.

      The disableCoverage group is a special group used by code coverage writers to ensure coverage tools are disabled for
      tests that are performance-critical.
      -->
      <groups inherit="true" groups="performance, disableCoverage"/>

      <!-- Specify the list of modes this test can be run in.

      Like test ids, mode names are usually TitleCase, with multiple dimensions delimited by an ``_`` underscore,
      e.g. ``CompressionGZip_AuthNone``.

      Test modes are configured with a Python lambda that returns a list of modes, using a single parameter which is
      an instance of ``pysys.config.descriptor.TestModesConfigHelper`` providing access to the inherited modes (and other
      useful functions/fields). Each mode in the returned list is defined by a dictionary containing parameters to be set
      on the test object and/or a ``mode`` name. If the mode name is not explicitly provided, a default mode name is
      generated by concatenating the parameter values with ``_`` (with a ``paramName=`` prefix for any numeric/boolean
      values).

      In your Python lambda you can return a simple list of modes, or combine your own modes with inherited modes
      defined by parent pysysdirconfigs. You can also use the power of Python list comprehensions to exclude certain modes,
      perhaps based on dynamic information such as the operating system. Project properties can be accessed
      using ``helper.project.PROPERTY_NAME``. Avoid expensive operations such as reading the file system from your lambda
      if possible.

      The first listed mode is the "primary" mode, which is the one that is used by default when executing tests with
      no ``-m`` argument.

      It's often useful to combine multiple mode 'dimensions', for example all the combinations of a list of web browsers
      with a list of databases, or compression methods and authentication types. Rather than writing out every combination
      manually, you can use the function ``helper.createModeCombinations`` to automatically generate all combinations.

      Modes can be used to define similar subtests that share the same test class logic, for example testing your
      application's output when given various different input test vectors. For this common use case, if you already (or
      plan to) define multiple execution modes inherited in a parent directory, you usually want to
      use ``helper.createModeCombinations(helper.inheritedModes, [...])`` in your test so that each of the subtests you
      define in that second argument are executed in each of the inherited modes (if any).

      A test can use self.mode to find out which mode it is executing and/or self.mode.params to access any parameters.

      The specified Python code is invoked using `pysys.utils.safeeval.safeEval`.

      -->
      <modes>
              lambda helper: [
                              mode for mode in

```

--------------------------------

### CollectTestOutputWriter Configuration Example

Source: https://pysys-test.github.io/pysys-test/autodocgen/pysys.writer.testoutput.html

Example of configuring the CollectTestOutputWriter using properties in the project configuration. This example shows how to set includeTestIf to collect output only from unit tests or local developer runs.

```xml
<property name="includeTestIf">lambda testObj:
        'unitTest' in testObj.descriptor.groups
        or testObj.project.getProperty('isLocalDeveloperTestRun',False)
</property>
```

--------------------------------

### CSV Format Example

Source: https://pysys-test.github.io/pysys-test/autodocgen/pysys.writer.outcomes.html

Example of the CSV column output format for test results.

```text
id, title, cycle, startTime, duration, outcome
```

--------------------------------

### Initialize Test Container and Output Directory

Source: https://pysys-test.github.io/pysys-test/_modules/pysys/baserunner.html

Sets up the test environment, including configuring logging handlers, determining the output directory, and cleaning up previous runs. This is invoked when a test container is called.

```python
def __call__(self, *args, **kwargs):
    exc_info = []
    self.testStart = time.time()
    defaultLogHandlersForCurrentThread = pysysLogHandler.getLogHandlersForCurrentThread()
    try:
        try:
            self.testFileHandlerStdout = logging.StreamHandler(_UnicodeSafeStreamWrapper(self.testFileHandlerStdoutBuffer, writebytes=False, encoding='utf-8'))
            self.testFileHandlerStdout.setFormatter(self.runner.project.formatters.stdout)
            self.testFileHandlerStdout.setLevel(stdoutHandler.level)
            pysysLogHandler.setLogHandlersForCurrentThread(defaultLogHandlersForCurrentThread+[self.testFileHandlerStdout])

            if os.path.isabs(self.runner.outsubdir):
                self.outsubdir = os.path.join(self.runner.outsubdir, self.descriptor.id)
            else:
                self.outsubdir = os.path.join(self.descriptor.testDir, self.descriptor.output, self.runner.outsubdir)
                if self.descriptor.mode:
                    self.outsubdir += '~'+self.descriptor.mode

            try:
                if not self.runner.validateOnly: 
                    if self.runner.cycle <= 1: 
                        deletedir(self.outsubdir, onerror=TestContainer.__onDeleteOutputDirError)
                    else:
                        with global_lock:
                            if self.outsubdir not in TestContainer.__purgedOutputDirs:
                                deletedir(self.outsubdir, onerror=TestContainer.__onDeleteOutputDirError)
                                TestContainer.__purgedOutputDirs.add(self.outsubdir)
            except Exception as ex:
                raise Exception('Failed to clean test output directory before starting test: %s'%ex)
            
            if self.runner.cycle > 1: 
                self.outsubdir = os.path.join(self.outsubdir, 'cycle%d' % (self.cycle+1))
            mkdir(self.outsubdir)
            initialOutputFiles = os.listdir(toLongPathSafe(self.outsubdir))

            runLogEncoding = self.runner.getDefaultFileEncoding('run.log') or PREFERRED_ENCODING
            self.testFileHandlerRunLog = logging.StreamHandler(_UnicodeSafeStreamWrapper(
                io.open(toLongPathSafe(os.path.join(self.outsubdir, 'run.log')), 'a', encoding=runLogEncoding), 
                writebytes=False, encoding=runLogEncoding))
            self.testFileHandlerRunLog.setFormatter(self.runner.project.formatters.runlog)
            self.testFileHandlerRunLog.setLevel(logging.INFO)
            if stdoutHandler.level == logging.DEBUG: self.testFileHandlerRunLog.setLevel(logging.DEBUG)
            pysysLogHandler.setLogHandlersForCurrentThread(defaultLogHandlersForCurrentThread+[self.testFileHandlerStdout, self.testFileHandlerRunLog])

            self.runner.logTestHeader(self.descriptor, self.cycle)
            
            if initialOutputFiles and not self.runner.validateOnly:
                log.warning('Some directories from a previous run could not be deleted from the output directory before starting this test: %s', ', '.join(initialOutputFiles))
        except KeyboardInterrupt:
            self.runner.handleKbrdInt()
            raise
        except Exception:
            exc_info.append(sys.exc_info())
            
        logHandlers = pysysLogHandler.getLogHandlersForCurrentThread()
            
        with global_lock:
            BaseTest._currentTestCycle = (self.cycle+1) if (self.runner.cycle > 1) else 0 
            try:

```
Tokens:
Raw