#+TODO: TODO STARTED FAILED | PASS SKIPPED
# The vertical bar indicates which states are final states.
#
# Using PASS instead of DONE because a test that suceeded is rather PASS than
# DONE. "PASS or FAIL" looks better than "DONE or FAIL".

* TODO Tests plan for cwdaemon version X
Version of template: 2024.06.03.
** TODO <Machine/OS identifier>
<Additional info about machine, OS, libs, compiler>
OS:
C compiler:
libc:
unixcw/libcw:

*** TODO tests of build system
**** TODO make distcheck
*Steps*

Run the steps in main directory of project.

On FreeBSD you may need to use "gmake" (GNU make) instead of FreeBSD's
default "make".

1. ./configure
2. make distcheck

*Expected result*
1. Distribution package is built correctly (no errors during creation of
   archive).
2. Automatic tests executed during 'make distcheck' pass.
3. The distribution package contains all files intended for distribution (use
   your best judgement here).

*Actual result*

**** TODO ./configure flags

Test that all flags supported by ./configure script produce "buildable" code.

*Steps*
1. Run "make clean && ./configure && make && make check",
2. Run "make clean && ./configure --enable-functional-tests && make && make check",
3. Run "make clean && ./configure --enable-functional-tests --enable-long-functional-tests && make && make check",

*Expected result*

Each of the above tests results in correct build: binaries are built
correctly, including binaries for all kinds of tests.

You don't have to do full tests with these test binaries, just check that
they are there.

*Actual result*

**** TODO regression with make distcheck

Confirm that failure of 'make distcheck' after specific sequence of steps does not occur.

On Alpine 3.17 and with certain version of tests/unit_tests/Makefile.am, the
steps ended with compilation error, and distribution package was not created.

The error message was:

gcc  -g -O2 -Wall -Wextra -pedantic -Wswitch-default -Wimplicit-function-declaration -Wswitch-enum -Wfloat-equal -Wpointer-arith -Wcast-qual -Wwrite-strings -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wredundant-decls -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict -Wnull-dereference -Wjump-misses-init -Wdouble-promotion -Wshadow -Wformat=2 -std=c99   -o utils ../../src/utils-utils.o utils-utils.o  
/usr/lib/gcc/x86_64-alpine-linux-musl/12.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find utils-utils.o: No such file or directory
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:328: utils] Error 1
make[4]: Leaving directory '/home/acerion/cwdaemon/git_repo/cwdaemon-0.12.0/_build/sub/tests/unit_tests'



*Steps*
make distclean
autoreconf
automake
./configure
make
make check
make distcheck  # <---- In this step a compilation error was occurring


*Expected result*
Distribution archive is created correctly.

*Actual result*

*** TODO unit tests

Execute tests implemented in tests/unit_tests.

*Steps*
1. ./configure --enable-functional-tests
2. make check
3. Wait for all unit tests to be executed. You can terminate the tests as
   soon as unit tests complete, and functional tests start being executed.

*Expected result*
All unit tests should have "PASS" status in output of "make check" command.

This shows examples of logs for unit tests (notice the "unit_tests"
directory):
PASS: unit_tests/daemon_utils
PASS: unit_tests/daemon_options
PASS: unit_tests/tests_random
PASS: unit_tests/tests_time_utils

*Notes*

Count of unit tests depends on whether you pass "--enable-functional-tests"
to ./configure or not. Test both cases.

*Actual result*

 - With functional tests not enabled:

 - With functional tests enabled:

*** TODO fuzzing tests
**** TODO simple fuzzing test

*Purpose*

Run simple fuzzing test to catch memory errors.

Run the test early in testing phase, certainly before doing manual tests. Let
an unattended fuzzing test detect problems early.

*Preconditions*

1. Be sure to use large count of iterations in the test: set value of
   "n_iters" in test_run() to high value.

2. Be sure to run the test under valgrind: set "supervisor_id" to
   "supervisor_id_valgrind" in options passed to server_start().

3. Be sure to have all tests enabled in "g_test_cases[]".

4. Plug-in an USB-to-UART converter.


*Steps*

1. Compile cwdaemon package with fuzzing test enabled:

   ./configure --enable-functional-tests
   make
   make check

   You can interrupt the automatic execution of unit and functional tests.

2. Run test program: tests/fuzzing/simple/test_program
3. Look at summary of valgrind test, looking for information about memory
   errors.


*Expected result*

1. valgrind reports no memory leaks (with possible exception of memory leak
   described in ticket R0018).

2. valgrind reports no memory errors of other types.

3. cwdaemon's test framework doesn't report any errors on its own

*Actual result*

*** TODO functional tests

**** TODO unattended short tests

*Goal*
Run unattended tests, but without "long" variants. Run relatively short tests
and look at final result to have some indication of general status of tests
before delving deeper into testing procedures that require more attention.


*Preconditions*
1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests
3. make


*Steps*
1. Run "make check"
2. Let the unattended tests run and complete


*Expectations*
The unattended tests executed through "make check" pass. We can now perform
tests that require more attention from tester.


*Actual result*

**** TODO unattended long tests

*Goal*
Run unattended tests, but this time also with "long" variants. Run relatively
long tests and look at final result to have some indication of general status
of tests before delving deeper into testing procedures that require more
attention.


*Preconditions*
1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests
3. make


*Steps*
1. Run "make check"
2. Let the unattended tests run and complete


*Expectations*
The unattended tests executed through "make check" pass. We can now perform
tests that require more attention from tester.


*Actual result*

**** TODO PLAIN request

*Preconditions*
1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests
3. make && make check
4. Interrupt the build process when tests start being executed


*Test program*
./tests/functional_tests/unattended/request_plain/test_program


*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test


*Expectations*
Final result of the test is a "PASS"


*Actual result*

**** TODO CARET request

*Preconditions*
1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests
3. make && make check
4. Interrupt the build process when tests start being executed


*Test program*
tests/functional_tests/unattended/request_caret/test_program


*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test


*Expectations*
Final result of the test is a "PASS"


*Actual result*

**** TODO Escape requests

***** TODO RESET        '0'

*Preconditions*

1. Connect USB-to-UART converter.


*Info*

The following parameters are reset with RESET Escape request:
 - Morse speed,
 - tone (frequency) of played Morse code,
 - volume of played Morse code,
 - sound system used to play Morse code,
 - PTT delay
 - weighting of Morse code,
 - state of libcw,
 - word mode (to false),
 - state of pins of cwdevice,
 - cwdaemon's internal PTT flags


*Steps*
1. in terminal A start cwdaemon with specific values of some of parameters.
   Those will be the initial/default parameters.

   ./src/cwdaemon -n -x s --wpm 15 --tone 1000 --volume 100 --weighting 50

2. In terminal B start "nc" as a client of cwdaemon:

   nc -u 127.0.0.1 6789

3. In terminal B send some text to confirm that cwdaemon can play it with
   initial/default parameters.

4. In terminal B send few Escape requests to modify parameters of cwdaemon:
   - Morse speed: change to 8 (Esc 2)
   - tone: change to 400 (Esc 3)
   - volume: change to 20 (Esc g)
   - weighting: change to 0 (Esc 7)

5. In terminal B send some text to confirm that cwdaemon can play it with
   modified parameters.

6. In terminal B send RESET Escape request to change cwdaemon's parameters
   back to their initial/default values.

7. In terminal B send some text to confirm that cwdaemon is playing the text
   with initial/default parameters.


*Expected result*
1. RESET Escape sent in step 6 does reset parameters to their initial values,
   which is confirmed by how cwdaemon plays a request in step 7.

*Actual result*

***** TODO SPEED        '2'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc2.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_esc2.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors

*Actual result*

***** TODO TONE         '3'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc3.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_esc3.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors

*Actual result*

***** TODO ABORT        '4'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc64.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_esc64.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors
3. Text played by cwdaemon can be interrupted by ABORT Escape request when
   cwdaemon works in interruptible mode.
4. Text played by cwdaemon cannot be interrupted by ABORT Escape request when
   cwdaemon works in non-interruptible mode.
5. cwdaemon behaves as described in points 3 and 4 even when invalid ABORT
   Escape request is being sent, i.e. when the value of the request is
   non-empty.

*Actual result*

***** TODO EXIT         '5'

*Preconditions*
1. Connect USB-to-UART converter.

*Test program*
./tests/functional_tests/unattended/request_esc_exit/test_program

*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*

***** TODO WORD_MODE    '6'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc64.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_esc64.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors
3. cwdaemon is being switched between word-mode (non-interruptible mode) and
   regular mode (interruptible mode).

   A current mode can be recognized by cwdaemon's response to ABORT Escape
   request.

   Played text cannot (in word-mode) or can (in regular mode) be interrupted
   by ABORT Escape request.

*Actual result*

***** TODO WEIGHTING    '7'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc7.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_esc7.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. Weighting of dits and dashes changes as expected,
3. cwdaemon doesn't report any unexpected errors.

*Actual result*

***** TODO CWDEVICE     '8'

*Preconditions*
1. Attach two USB-to-UART devices acting as two cwdevices: default one, and
   non-default one.

*Test program*
./tests/functional_tests/unattended/request_esc_cwdevice/test_program

*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*

***** TODO PORT         '9'

*Preconditions*
1. Connect USB-to-UART converter.

*Test program*
./tests/functional_tests/supervised/request_esc_port/test_program

*Steps*
1. Manually start the test program
2. Read the banner text displayed at the beginning of test,
3. Observe the logs displayed during execution of test,
4. Wait for final status of the test.

*Expectations*
1. Final result of the test is a "PASS".
2. None of characteristics of played sounds are changing between iterations
   of test.
3. You can see the following message in logs of cwdaemon when PORT Escape
   request is sent to cwdaemon:

   [WW] cwdaemon: obsolete request "9" (change network port), ignoring


*Actual result*

***** TODO PTT_STATE    'a'

*Preconditions*

1. Prepare a multimeter or logic state analyzer.
2. Read information at the beginning of tests/cwtest_esca.pl script about 3
   modes.
3. Connect USB-to-UART converter.
4. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s --cwdevice <cwdevice>
5. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esca.pl

*Steps*
1. Connect the multimeter to PTT pin of the USB-to-UART converter.
2. In terminal B run the test program

   perl ./cwtest_esca.pl --mode X

   Do this with value of mode (X) equal to 0, 1, and 2.
3. Observe output of test in terminal B
4. Observe readings on the multimeter, showing state of PTT pin (ON or OFF).


*Expected result*
1. State of PTT pin changes at all, as shown by multimeter's reading,
2. State of PTT pin changes according to mode used by the test script.

*Actual result*

***** TODO SSB_WAY      'b'

*Expected result*
Test passes


*Actual result*

***** TODO TUNE         'c'
****** TODO Perl script

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escc.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_escc.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B
4. Count the duration of each generated tuning tone.


*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors,
3. Each generated tuning tone lasted for expected count of seconds.


*Actual result*

****** TODO Manual test, with observing PTT Flag

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s -iii -t 1
3. In another terminal (terminal B) run nc:
   nc -u 127.0.0.1 6789


*Steps*
1. In terminal B request long tuning time
   ^[c9
2. In terminal A observe that PTT pin is raised:

   [II] cwdaemon: -------------------
   [II] cwdaemon: received Escape request: "<ESC>c" / "<ESC>0x63"
   [II] cwdaemon: requested tuning time [s]: "9"
   [II] cwdaemon: PTT (TUNE) on
   [DD] cwdaemon: PTT flag +PTT_ACTIVE_AUTO (0x01/Ame)
   [DD] cwdaemon: keying event 1

3. After the tuning time elapses, observe in terminal A that PTT pin is
   lowered:

   [II] cwdaemon: low TQ callback: start, TQ len = 1, PTT flag = 0x01/Ame
   [II] cwdaemon: low TQ callback: branch 1, PTT flag = 0x01/Ame
   [DD] cwdaemon: PTT flag = 0 (0x00/ame)
   [II] cwdaemon: PTT (auto) off
   [II] cwdaemon: low TQ callback: end, TQ len = 1, PTT flag = 0x00/ame
   [DD] cwdaemon: keying event 0


*Expected result*
1. PTT pin is raised and lowered as expected,
2. cwdaemon doesn't report any unexpected errors


*Actual result*

***** TODO TX_DELAY     'd'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escd.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_escd.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors.

*Actual result*

***** TODO BAND_SWITCH  'e'

*Expected result*
Test passes


*Actual result*

***** TODO SOUND_SYSTEM 'f'

The test may be trying to switch to invalid sound system "?"/"<NONE>". Such
attempts will be ignored by cwdaemon, and cwdaemon SHOULD continue to use
previous sound system.

*Preconditions*
1. Attach USB-to-UART device acting as cwdevice.

*Test program*
./tests/functional_tests/supervised/request_esc_sound_system/test_program

*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*

***** TODO VOLUME       'g'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escg.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_escg.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. Volume of sounds produced by cwdaemon changes as expected,
3. cwdaemon doesn't report any unexpected errors.

*Actual result*

***** TODO REPLY        'h'

*Test program*
./tests/functional_tests/unattended/request_esc_reply/test_program

*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*

**** TODO Command line options

***** TODO -h, --help

***** TODO -V, --version

***** TODO -d, --cwdevice <device>

*Preconditions*
1. Connect two cwdevices


*Steps*
1. In one terminal (terminal A) start a test instance of cwdaemon:

   ./src/cwdaemon -n -x s --cwdevice <cwdevice-1>

2. In another terminal (terminal B) send some text using nc. As the text is
   being played, confirm that cwdaemon is toggling "keying" pin on
   <cwdevice-1>, but not in <cwdevice-2>.

3. In terminal A stop cwdaemon and start it again, but this time with the
   second cwdevice.

   ./src/cwdaemon -n -x s --cwdevice <cwdevice-2>

4. In another terminal (terminal B) send some text using nc. As the text is
   being played, confirm that cwdaemon is toggling "keying" pin on
   <cwdevice-2>, but not in <cwdevice-1>.


*Expected result*
1. cwdaemon is using the right cwdevice, as specified in command line.


*Actual result*

***** TODO -o, --options <option>

*Test program*
./tests/functional_tests/unattended/option_cwdevice_tty_lines/test_program

*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test
4. Manually run at least one test with the long "--options" form of the
   command line option.
5. Try passing invalid values of "key=" or "ptt=" keywords in command line.


*Expectations*
1. Final result of the test is a "PASS".
2. Handling of long "--options" option work.
3. Invalid values of "key=" or "ptt=" keywords are rejected.


*Actual result*

***** TODO -n, --nofork

*Preconditions*

Find out the syslog output to which daemon which output a "daemon" processes
are logging.

On FreeBSD, per /etc/syslog.conf, the INFO messages go to
/var/log/daemon.log.

On Linux, you will be able to see messages in output of "journalctl -f":
<netkeyer[5093]: received request: "is this syslog?">

****** TODO do not fork

*Steps*

1. Run cwdaemon in non-forking mode

   ./src/cwdaemon [-n/--nofork] -x p -ii

2. In another terminal send some PLAIN message to cwdaemon.


*Expected result*

1. Observe that cwdaemon is still attached to current terminal (it didn't go
   to background).
2. Observe that cwdaemon is logging messages to stdout


*Actual result*

****** TODO do fork

*Steps*

1. Confirm that no other cwdaemon is already running

   ps -A | grep cwdaemon

2. Run cwdaemon in forking mode

   ./src/cwdaemon -x p -ii

3. Confirm that cwdaemon process is present with this command:

   ps -A | grep cwdaemon

4. In another terminal send some PLAIN message to cwdaemon.

5. In another terminal send EXIT Escape request (<ESC>5).

6. Confirm that cwdaemon process is no longer present (because it exited)
   with this command:

   ps -A | grep cwdaemon


*Expected result*

1. After step 2 the cwdaemon is no longer attached to current terminal (it
   went to background).
2. In step 4 cwdaemon is not logging messages to stdout.
3. In step 4 cwdaemon is logging messages to syslog instead.
4. In step 4 cwdaemon is playing Morse code using sound system.


*Actual result*

***** TODO -p, --port <port>

*Test program*
./tests/functional_tests/unattended/option_port/test_program


*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test


*Expectations*
Final result of the test is a "PASS"


*Actual result*

***** TODO -s, --wpm <speed>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.


*Actual result*

***** TODO -t, --pttdelay <time>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon.
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*

***** TODO -x, --system <sound system>

*Preconditions*
1. Connect a cwdevice


*Info*
cwdaemon may fail to run with OSS on Linux, and detecting the failure by
cwdaemon may last ~15 seconds.


*Steps*
1. Run cwdaemon with different sound systems: one of 'n', 'c', 'o', 'a', 'p',
   's'.
2. Run cwdaemon with invalid sound system, e.g. 'u'.
3. Observe if cwdaemon exits due to unsupported sound system, or if it starts
   correctly.
4. If it starts correctly, try to sending some text through "nc".


*Expected results*
1. cwdaemon doesn't start correctly if a sound system not supported on the
   test platform is specified.
2. cwdaemon starts correctly if a sound system supported on the test platform
   is specified.
3. If cwdaemon starts correctly, it plays a text using specified sound
   system.
4. If cwdaemon starts correctly, Morse code is keyed on cwdevice's 'key' pin.
5. Both short and long form of the command line option works the same.


*Actual result*

***** TODO -v, --volume <volume>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*

***** TODO -w, --weighting <weight>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.
4. cwdaemon accepts negative values (you may need to put them in quotes)

*Actual result*

***** TODO -T, --tone <tone>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*

***** TODO -i

*Preconditions*

1. Connect USB-to-UART converter.


*Steps*

1. In one terminal (terminal A) start cwdaemon without "-i" flags:

   ./src/cwdaemon -n -x s -d <cwdevice>

2. In another terminal (terminal B) send a valid TONE Escape request
   (<ESC>3), send an invalid TONE Escape request, and then play some text.

3. In terminal A observe that with no "-i" flags specified in command line,
   cwdaemon prints only errors and warnings

4. In terminal A do a series of starts of cwdaemon with increasing amount of
   "-i" flags. In terminal B send a TONE Escape request, then play some text,
   and then in terminal A observe increasing verbosity of cwdaemon's logs.

   "-i"     ->  notice
   "-ii"    ->  info
   "-iii"   ->  debug
   "-iiii"  ->  still debug


*Expected result*
1. Verbosity of logs in cwdaemon increases with count of "i" options passed
   in command line.


*Actual result*

***** TODO -y, --verbosity <threshold>

*Preconditions*

1. Connect USB-to-UART converter.

*Steps*

1. In one terminal (terminal A) start cwdaemon without "-y"/"--verbosity"
   flag:

   ./src/cwdaemon -n -x s -d <cwdevice>

2. In another terminal (terminal B) send a valid TONE Escape request
   (<ESC>3), send an invalid TONE Escape request, and then play some text.

3. In terminal A observe that with no "-y"/"--verbosity" flag specified in
   command line, cwdaemon prints only errors and warnings.

4. In terminal A do a series of starts of cwdaemon with increasing a
   verbosity level passed to "-y"/"--verbosity" flag (n (none), e (error), w
   (warning), i (information), d (debug)). In terminal B send a TONE Escape
   request, then play some text, and then in terminal A observe increasing
   verbosity of cwdaemon's logs.

   In particular confirm that with "n" flag cwdaemon doesn't even print
   errors.


*Expected result*

1. Verbosity of logs in cwdaemon increases with level of verbosity passed to
   "-y"/"--verbosity" in command line.


*Actual result*

***** TODO -I, --libcwflags <flags>

*Goal*
Confirm that option that specifies debug flags for libcw is working.


*Info*
In this test cwdaemon is executed:
 - in non-daemonized mode to observe console logs;
 - with null keying device because cwdevice is not essential for this test.


*Steps*

1. In terminal A, open connection to cwdaemon.

   nc -u localhost 6789
   OR
   nc -u 127.0.0.1 6789

2. In terminal B, run cwdaemon without the tested flag. Notice the 'd'
   (DEBUG) threshold for cwdaemon.

   ./src/cwdaemon -d null -n -x s -y d

3. In terminal A, send some characters to cwdaemon with nc.

4. Confirm that cwdaemon's log output shows cwdaemon logs, but doesn't show
   any libcw logs (because '--libcwflags' option is not provided).

5. Kill cwdaemon started in step 2. Start cwdaemon with '--libcwflags'
   option. Notice the 'd' (DEBUG) threshold for cwdaemon.

   ./src/cwdaemon --libcwflags=4294967295 -d null  -n -x p -y d

6. Send characters to cwdaemon using nc.

7. Confirm that cwdaemon's log output shows cwdaemon logs and show libcw logs
   with severity DEBUG or higher.

8. Kill cwdaemon started in step 4. Start cwdaemon with '--libcwflags'
   option. Notice the 'w' (WARNING) threshold for cwdaemon.

   ./src/cwdaemon --libcwflags=4294967295 -d null  -n -x p -y w

9. Send characters to cwdaemon using nc. Also send VOLUME Escape request
   (<ESC>g) with value out of range to trigger logging errors by cwdaemon.

10. Confirm that cwdaemon's log output shows cwdaemon logs. If there are any
    libcw logs, the logs have severity only WARNING or ERROR.


*Expected result*

1. libcw logs are visible in cwdaemon's log output only if `--libcwflags`
   flag is used

2. Threshold for libcw logs depends on value of '-y' flag passed to cwdaemon.

*Actual result*

***** TODO -f, --debugfile <output>

*Preconditions*

1. Connect USB-to-UART converter.


*Steps*

1. Confirm that cwdaemon logs messages to text file when started like this in
   non-forking mode:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] /tmp/non_forking.txt

2. Confirm that cwdaemon logs messages to text file when started like this in
   forking mode:

   ./src/cwdaemon -x s -d /dev/cuaU0 -y d [-f/--debugfile] /tmp/forking.txt

3. Confirm that cwdaemon logs messages to stdout when started like this:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] stdout 2>/tmp/stderr.txt

4. Confirm that cwdaemon logs messages to stderr when started like this:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] stderr 1>/tmp/stdout.txt

5. Confirm that cwdaemon rejects "syslog" as output of logs when started like
   this:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] syslog


*Expected result*
1. All of the behaviours from "Steps" are observed in running cwdaemon.
2. All of the behaviours from "Steps" are the same for short and long form of
   the command line option.


*Actual result*

**** TODO Misc tests

***** TODO reset register callback

*Preconditions*

1. Connect USB-to-UART converter.
2. Compile cwdaemon with functional tests enabled

   ./configure --enable-functional-tests
   make
   make check


*Test program*
./tests/functional_tests/unattended/reset_register_callback/test_program


*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*

***** TODO State of tty DTR and RTS pins during inactivity

*Purpose*

Confirm that cwdaemon doesn't change initial state of tty cwdevice's DTR and
RTS pins right after start.

Incorrect initialization of tty device has led to
https://github.com/acerion/cwdaemon/issues/12.

You can use cwdaemon 0.10.2 as a baseline. In a version following 0.10.2 a
change in ttys_init() was made that introduced an undesired behaviour of the
pins.


*Prerequisites*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. A device capable of showing a logical state on converter's pins: a
   multimeter or logical state analyzer.


*Steps*

1. Connect USB-to-UART converter to USB port.

2. Measure initial state of DTR and RTS pins

   Expectation: the state should be "high".

3. Start cwdaemon, specify a proper cwdevice through "-d" option

   ./src/cwdaemon -n -x s -ii -d ttyUSB0

4. Measure again the state of DTR and RTS pins.

   Expectation: cwdaemon should not change the state of the pins during start
   of cwdaemon and initialization of the cwdevice. The state on both pins
   should be "high".

5. Send some text to be played by cwdaemon

   nc -u 127.0.0.1 6789
   <Enter some text>

   After text is played and keyed on cwdevice, measure state of DTR and RTS
   pins.

   Expectation: after the text is played and keyed on cwdevice, the state of
   both pins should be "high".

6. Send "reset" command to cwdaemon

   Send "<ESC>0" Escape request to cwdaemon.

   nc -u 127.0.0.1 6789
   <Press ESC key>
   <Press "0" key>
   <Press Enter key>


   In cwdaemon's logs look for confirmation that reset is being performed:

   [INFO ] cwdaemon: requested resetting of parameters
   [INFO ] cwdaemon: resetting completed


   Wait for 2 seconds for the end of reset, then measure state of DTR and RTS
   pins.

   Expectation: after a reset is completed, the state of both pins should be
   "high".

7. Terminate cwdaemon

   Either press Ctrl-C while cwdaemon is running in foreground, or send
   "<ESC>5" Escape request to cwdaemon.

   Then measure state of DTR and RTS pins.

   Expectation: after an exit is performed, the state of both pins should be
   "high".


*Expected result*

All expectations from above steps are met.

*Actual result*

***** TODO tests/cwtest.pl

*Preconditions*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. cwdaemon must be started (automatically or manually), running on default
   port, with <platform's preferred sound system>.

   ./src/cwdaemon -n -x s -d cuaU0


*Test program*
cwtest.pl


*Steps*
1. Enter "tests" directory
2. Start the test script

   perl ./cwtest.pl --cycles 2

3. Observe test's output and cwdaemon's output.


*Expected result*
1. There are no errors in messages printed by cwdaemon.
2. There are no errors in messages printed by the test.


*Actual result*

***** TODO tests/cwtest_short_space.pl

*Preconditions*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. cwdaemon must be started (automatically or manually), running on default
   port, with <platform's preferred sound system>.

   ./src/cwdaemon -n -x s -d cuaU0


*Test program*
cwtest_short_space.pl


*Steps*
1. Enter "tests" directory
2. Start the test script

   perl ./cwtest_short_space.pl --cycles 20

3. Observe test's output and cwdaemon's output.


*Expected result*
1. There are no errors in messages printed by cwdaemon.
2. There are no errors in messages printed by the test.


*Actual result*

*** TODO gcov/lcov

*Purpose*

Confirm that it's possible to generate a coverage report for unit tests. It
is not expected that specific level of coverage is achieved.

Make sure that you can see coverage reports for source code of cwdaemon, i.e.
for files from ROOT/src.


*Prerequisites*

1. "lcov" package is installed.
2. The compilation and the coverage must be done with the same toolchain, and
   therefore on FreeBSD you may need to do this:

   export CC=clang

3. Connect an USB-to-UART cwdevice


*Steps*

The steps are copied from relevant sections in top-level README file:

1. ./configure --enable-functional-tests --enable-gcov
2. make gcov
3. <web browser> tests/coverage/unit_tests/daemon_<functionality>/index.html
4. make clean


*Expected results*

1. Summary of configuration step correctly shows enabled gcov,
2. There are no errors during generation of gcov/lcov report,
3. Contents of the report looks correct (no obvious problems with the
   report).
4. There are no errors during cleanup step.


*Actual result*

*** TODO memory tests with valgrind
**** TODO memory tests with valgrind for functional/unattended tests

*Goal*

Run all of cwdaemon's unattended functional tests in a setup where cwdaemon is
running under control of valgrind.


*Preconditions*

1. Connect USB-to-UART converter (a cwdevice).


*Steps*

1. Enable compilation of unattended functional tests

   ./configure --enable-functional-tests --enable-long-functional-tests

2. Compile cwdaemon

   make && make check

   You can terminate the "make check" command as soon as build system starts
   executing unit tests.

3. Prepare environment variables that will be used by all test programs
   executed with "make check"

   export CWDAEMON_TEST_SOUND_SYSTEM=null
   export CWDAEMON_TEST_SUPERVISOR=valgrind

4. Find all unattended tests

   find tests/functional_tests/unattended/ -name test_program

5. Run each binary by hand, one at a time.

   The purpose of running each test at a time is to get informed about a
   possible failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

**** TODO memory tests with valgrind for functional/supervised tests

*Goal*

Run all of cwdaemon's supervised functional tests in a setup where cwdaemon
is running under control of valgrind.


*Preconditions*

1. Connect USB-to-UART converter (a cwdevice).


*Steps*

1. Enable compilation of supervised functional tests

   ./configure --enable-functional-tests --enable-long-functional-tests

2. Compile cwdaemon

   make && make check

   You can terminate the "make check" command as soon as build system starts
   executing unit tests.

3. Prepare environment variables that will be used by all test programs
   executed with "make check"

   export CWDAEMON_TEST_SOUND_SYSTEM=null
   export CWDAEMON_TEST_SUPERVISOR=valgrind

4. Find all supervised tests

   find tests/functional_tests/supervised/ -name test_program

5. Run each binary by hand, one at a time.

   The purpose of running each test at a time is to get informed about a
   failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

**** TODO memory tests with valgrind for fuzzing tests

*Goal*

Run all of cwdaemon's fuzzing tests in a setup where cwdaemon is running
under control of valgrind.


*Steps*

1. Confirm that fuzzing tests done few sections above were already executed
   with cwdaemon being under control of valgrind.

*Expected result*

No additional test is needed here because the primary fuzzing tests already
use valgrind.

*Actual result*

**** TODO memory tests with valgrind for unit tests

*Goal*

Run all of cwdaemon's unit tests in a setup where unit tests are running
under control of valgrind.


*Steps*

1. Enable compilation of unit tests (unit tests of cwdaemon only)

   make clean
   ./configure

2. Compile cwdaemon and unit tests

   make && make check

3. Find all unit test binaries

   find tests/unit_tests/* -type f  -executable

4. Run each binary by hand, one at a time.

   /usr/local/bin/valgrind -s --leak-check=full --show-leak-kinds=all --track-fds=yes <path to test binary>

   The purpose of running each test at a time is to get informed about a
   failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

