#+TODO: TODO STARTED FAILED | PASS SKIPPED
# The vertical bar indicates which states are final states.
#
# Using PASS instead of DONE because a test that suceeded is rather PASS than
# DONE. "PASS or FAIL" looks better than "DONE or FAIL".

* PASS Tests plan for cwdaemon version 0.13.0
Version of template: 2024.06.03.
** PASS FreeBSD 14.1-RELEASE
<Additional info about machine, OS, libs, compiler>
OS: FreeBSD 14.1-RELEASE
C compiler: gcc 13.2.0
libc:
unixcw/libcw: unixcw 3.6.1, libcw 8:0:0

The overall status of tests for FreeBSD is PASS even though gcov/lcov tests
have failed. This has been addressed in comment at the end of the failed
test.

*** PASS tests of build system
**** PASS make distcheck
*Steps*

Run the steps in main directory of project.

On FreeBSD you may need to use "gmake" (GNU make) instead of FreeBSD's
default "make".

1. ./configure
2. make distcheck

*Expected result*
1. Distribution package is built correctly (no errors during creation of
   archive).
2. Automatic tests executed during 'make distcheck' pass.
3. The distribution package contains all files intended for distribution (use
   your best judgement here).

*Actual result*

PASS. I needed to run the test with GNU make rather than FreeBSD's default
make.

**** PASS ./configure flags

Test that all flags supported by ./configure script produce "buildable" code.

*Steps*
1. Run "make clean && ./configure && make",
2. Run "make clean && ./configure --enable-functional-tests && make && make check",
3. Run "make clean && ./configure --enable-functional-tests --enable-long-functional-tests && make && make check",

*Expected result*

Each of the above tests results in correct build: binaries are built
correctly, including binaries for all kinds of tests.

You don't have to do full tests with these test binaries, just check that
they are there.

*Actual result*

PASS: Binaries are being built.

The binaries are being built correctly.

**** PASS regression with make distcheck

Confirm that failure of 'make distcheck' after specific sequence of steps does not occur.

On Alpine 3.17 and with certain version of tests/unit_tests/Makefile.am, the
steps ended with compilation error, and distribution package was not created.

The error message was:

gcc  -g -O2 -Wall -Wextra -pedantic -Wswitch-default -Wimplicit-function-declaration -Wswitch-enum -Wfloat-equal -Wpointer-arith -Wcast-qual -Wwrite-strings -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wredundant-decls -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict -Wnull-dereference -Wjump-misses-init -Wdouble-promotion -Wshadow -Wformat=2 -std=c99   -o utils ../../src/utils-utils.o utils-utils.o  
/usr/lib/gcc/x86_64-alpine-linux-musl/12.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find utils-utils.o: No such file or directory
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:328: utils] Error 1
make[4]: Leaving directory '/home/acerion/cwdaemon/git_repo/cwdaemon-0.12.0/_build/sub/tests/unit_tests'


*Steps*
make distclean
autoreconf
automake
./configure
make
make check
make distcheck  # <---- In this step a compilation error was occurring


*Expected result*
Distribution archive is created correctly.

*Actual result*

PASS: there is no error during "gmake distcheck" (notice the usage of "gmake").

*** PASS unit tests

Execute tests implemented in tests/unit_tests.

*Steps*
1. ./configure --enable-functional-tests
2. make check
3. Wait for all unit tests to be executed. You can terminate the tests as
   soon as unit tests complete, and functional tests start being executed.

*Expected result*
All unit tests should have "PASS" status in output of "make check" command.

This shows examples of logs for unit tests (notice the "unit_tests"
directory):
PASS: unit_tests/daemon_utils
PASS: unit_tests/daemon_options
PASS: unit_tests/tests_random
PASS: unit_tests/tests_time_utils

*Notes*

Count of unit tests depends on whether you pass "--enable-functional-tests"
to ./configure or not. Test both cases.

*Actual result*

Unit tests PASS:

 - With functional tests not enabled:
PASS: unit_tests/daemon_utils
PASS: unit_tests/daemon_options
PASS: unit_tests/daemon_sleep

 - With functional tests enabled:
PASS: unit_tests/daemon_utils
PASS: unit_tests/daemon_options
PASS: unit_tests/daemon_sleep
PASS: unit_tests/tests_random
PASS: unit_tests/tests_string_utils
PASS: unit_tests/tests_time_utils
PASS: unit_tests/tests_morse_receiver
PASS: unit_tests/tests_events

*** PASS fuzzing tests
**** PASS simple fuzzing test

Run simple fuzzing test: tests/fuzzing/simple/test_program.

Run the test early in testing phase, certainly before doing manual tests. Let
an unattended fuzzing test detect problems early.

*Preconditions*

1. Be sure to use large count of iterations in the test: set value of
   "n_iters" in test_run() to high value.

2. Be sure to run the test under valgrind: set "supervisor_id" to
   "supervisor_id_valgrind" in options passed to server_start().

3. Be sure to have all tests enabled in "g_test_cases[]".

*Steps*
1. ./configure --enable-functional-tests
2. make && make check # (you may want to interrupt functional tests here)
3. ./tests/fuzzing/simple/test_program


*Expected result*

1. valgrind reports no memory leaks (with possible exception of memory leak
   described in ticket R0018).

2. valgrind reports no memory errors of other types.

3. cwdaemon's test framework doesn't report any errors on its own

*Actual result*

==80129== 
==80129== FILE DESCRIPTORS: 3 open (3 std) at exit.
==80129== 
==80129== HEAP SUMMARY:
==80129==     in use at exit: 14,811 bytes in 28 blocks
==80129==   total heap usage: 92 allocs, 64 frees, 1,616,699 bytes allocated
==80129== 
==80129== 2 bytes in 1 blocks are still reachable in loss record 1 of 18
==80129==    at 0x484CDE4: malloc (vg_replace_malloc.c:446)
==80129==    by 0x534455C: strdup (in /lib/libc.so.7)
==80129==    by 0x5272152: ??? (in /lib/libc.so.7)
==80129==    by 0x53445C6: ??? (in /lib/libc.so.7)
==80129==    by 0x53447C0: strerror (in /lib/libc.so.7)
==80129==    by 0x40537D: tty_probe_cwdevice (ttys.c:100)
==80129==    by 0x40441B: cwdaemon_cwdevice_set (cwdaemon.c:2411)
==80129==    by 0x405AAA: cwdaemon_option_cwdevice (options.c:148)
==80129==    by 0x403A13: cwdaemon_handle_escaped_request (cwdaemon.c:1048)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== 5 bytes in 1 blocks are still reachable in loss record 2 of 18
==80129==    at 0x484CDE4: malloc (vg_replace_malloc.c:446)
==80129==    by 0x534455C: strdup (in /lib/libc.so.7)
==80129==    by 0x5272132: ??? (in /lib/libc.so.7)
==80129==    by 0x53445C6: ??? (in /lib/libc.so.7)
==80129==    by 0x53447C0: strerror (in /lib/libc.so.7)
==80129==    by 0x40537D: tty_probe_cwdevice (ttys.c:100)
==80129==    by 0x40441B: cwdaemon_cwdevice_set (cwdaemon.c:2411)
==80129==    by 0x405AAA: cwdaemon_option_cwdevice (options.c:148)
==80129==    by 0x403A13: cwdaemon_handle_escaped_request (cwdaemon.c:1048)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== 20 bytes in 1 blocks are definitely lost in loss record 3 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x558D67D: pthread_cond_init (in /lib/libthr.so.3)
==80129==    by 0x486A818: cw_tq_new_internal (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4867F61: cw_gen_new (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4864B8D: cw_generator_new_internal (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4864CCF: cw_generator_new (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x403426: cwdaemon_open_libcw_output (cwdaemon.c:693)
==80129==    by 0x4035AD: cwdaemon_reset_libcw_output (cwdaemon.c:772)
==80129==    by 0x4035AD: cwdaemon_reset_almost_all (cwdaemon.c:666)
==80129==    by 0x4024AD: main (cwdaemon.c:2249)
==80129== 
==80129== 20 bytes in 1 blocks are definitely lost in loss record 4 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x558D67D: pthread_cond_init (in /lib/libthr.so.3)
==80129==    by 0x486A818: cw_tq_new_internal (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4867F61: cw_gen_new (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4864B8D: cw_generator_new_internal (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4864CCF: cw_generator_new (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x403426: cwdaemon_open_libcw_output (cwdaemon.c:693)
==80129==    by 0x403CA1: cwdaemon_handle_escaped_request (cwdaemon.c:1173)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== 20 bytes in 1 blocks are definitely lost in loss record 5 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x558D67D: pthread_cond_init (in /lib/libthr.so.3)
==80129==    by 0x486A818: cw_tq_new_internal (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4867F61: cw_gen_new (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4864B8D: cw_generator_new_internal (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4864CCF: cw_generator_new (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x403426: cwdaemon_open_libcw_output (cwdaemon.c:693)
==80129==    by 0x403FCF: cwdaemon_handle_escaped_request (cwdaemon.c:1178)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== 56 bytes in 1 blocks are still reachable in loss record 6 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x5272117: ??? (in /lib/libc.so.7)
==80129==    by 0x53445C6: ??? (in /lib/libc.so.7)
==80129==    by 0x53447C0: strerror (in /lib/libc.so.7)
==80129==    by 0x40537D: tty_probe_cwdevice (ttys.c:100)
==80129==    by 0x40441B: cwdaemon_cwdevice_set (cwdaemon.c:2411)
==80129==    by 0x405AAA: cwdaemon_option_cwdevice (options.c:148)
==80129==    by 0x403A13: cwdaemon_handle_escaped_request (cwdaemon.c:1048)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== 60 bytes in 3 blocks are definitely lost in loss record 7 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x558D67D: pthread_cond_init (in /lib/libthr.so.3)
==80129==    by 0x486A818: cw_tq_new_internal (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4867F61: cw_gen_new (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4864B8D: cw_generator_new_internal (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4864CCF: cw_generator_new (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x403426: cwdaemon_open_libcw_output (cwdaemon.c:693)
==80129==    by 0x4035AD: cwdaemon_reset_libcw_output (cwdaemon.c:772)
==80129==    by 0x4035AD: cwdaemon_reset_almost_all (cwdaemon.c:666)
==80129==    by 0x403789: cwdaemon_handle_escaped_request (cwdaemon.c:953)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== 64 bytes in 1 blocks are still reachable in loss record 8 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x5598762: ??? (in /lib/libthr.so.3)
==80129==    by 0x559140E: ??? (in /lib/libthr.so.3)
==80129==    by 0x559042E: ??? (in /lib/libthr.so.3)
==80129==    by 0x400AF5C: ??? (in /libexec/ld-elf.so.1)
==80129==    by 0x4009B88: ??? (in /libexec/ld-elf.so.1)
==80129==    by 0x4006DF8: ??? (in /libexec/ld-elf.so.1)
==80129== 
==80129== 64 bytes in 1 blocks are still reachable in loss record 9 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x5598762: ??? (in /lib/libthr.so.3)
==80129==    by 0x559140E: ??? (in /lib/libthr.so.3)
==80129==    by 0x558E28E: pthread_create (in /lib/libthr.so.3)
==80129==    by 0x4866259: cw_gen_start (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4034B4: cwdaemon_open_libcw_output (cwdaemon.c:709)
==80129==    by 0x4035AD: cwdaemon_reset_libcw_output (cwdaemon.c:772)
==80129==    by 0x4035AD: cwdaemon_reset_almost_all (cwdaemon.c:666)
==80129==    by 0x4024AD: main (cwdaemon.c:2249)
==80129== 
==80129== 64 bytes in 1 blocks are still reachable in loss record 10 of 18
==80129==    at 0x4852862: aligned_alloc (vg_replace_malloc.c:2263)
==80129==    by 0x5596DD5: ??? (in /lib/libthr.so.3)
==80129==    by 0x559685A: ??? (in /lib/libthr.so.3)
==80129==    by 0x5271CBB: ??? (in /lib/libc.so.7)
==80129==    by 0x53445C6: ??? (in /lib/libc.so.7)
==80129==    by 0x53447C0: strerror (in /lib/libc.so.7)
==80129==    by 0x40537D: tty_probe_cwdevice (ttys.c:100)
==80129==    by 0x40441B: cwdaemon_cwdevice_set (cwdaemon.c:2411)
==80129==    by 0x405AAA: cwdaemon_option_cwdevice (options.c:148)
==80129==    by 0x403A13: cwdaemon_handle_escaped_request (cwdaemon.c:1048)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== 64 bytes in 1 blocks are still reachable in loss record 11 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x5598762: ??? (in /lib/libthr.so.3)
==80129==    by 0x559140E: ??? (in /lib/libthr.so.3)
==80129==    by 0x558E28E: pthread_create (in /lib/libthr.so.3)
==80129==    by 0x4866259: cw_gen_start (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4034B4: cwdaemon_open_libcw_output (cwdaemon.c:709)
==80129==    by 0x403FCF: cwdaemon_handle_escaped_request (cwdaemon.c:1178)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== 100 bytes in 5 blocks are still reachable in loss record 12 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x558D67D: pthread_cond_init (in /lib/libthr.so.3)
==80129==    by 0x486A818: cw_tq_new_internal (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4867F61: cw_gen_new (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4864B8D: cw_generator_new_internal (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4864CCF: cw_generator_new (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x403426: cwdaemon_open_libcw_output (cwdaemon.c:693)
==80129==    by 0x4035AD: cwdaemon_reset_libcw_output (cwdaemon.c:772)
==80129==    by 0x4035AD: cwdaemon_reset_almost_all (cwdaemon.c:666)
==80129==    by 0x403789: cwdaemon_handle_escaped_request (cwdaemon.c:953)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== 192 bytes in 3 blocks are still reachable in loss record 13 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x5598762: ??? (in /lib/libthr.so.3)
==80129==    by 0x559140E: ??? (in /lib/libthr.so.3)
==80129==    by 0x558E28E: pthread_create (in /lib/libthr.so.3)
==80129==    by 0x4866259: cw_gen_start (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4034B4: cwdaemon_open_libcw_output (cwdaemon.c:709)
==80129==    by 0x4035AD: cwdaemon_reset_libcw_output (cwdaemon.c:772)
==80129==    by 0x4035AD: cwdaemon_reset_almost_all (cwdaemon.c:666)
==80129==    by 0x403789: cwdaemon_handle_escaped_request (cwdaemon.c:953)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== 1,664 bytes in 1 blocks are still reachable in loss record 14 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x55913FD: ??? (in /lib/libthr.so.3)
==80129==    by 0x559042E: ??? (in /lib/libthr.so.3)
==80129==    by 0x400AF5C: ??? (in /libexec/ld-elf.so.1)
==80129==    by 0x4009B88: ??? (in /libexec/ld-elf.so.1)
==80129==    by 0x4006DF8: ??? (in /libexec/ld-elf.so.1)
==80129== 
==80129== 1,664 bytes in 1 blocks are still reachable in loss record 15 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x55913FD: ??? (in /lib/libthr.so.3)
==80129==    by 0x558E28E: pthread_create (in /lib/libthr.so.3)
==80129==    by 0x4866259: cw_gen_start (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4034B4: cwdaemon_open_libcw_output (cwdaemon.c:709)
==80129==    by 0x4035AD: cwdaemon_reset_libcw_output (cwdaemon.c:772)
==80129==    by 0x4035AD: cwdaemon_reset_almost_all (cwdaemon.c:666)
==80129==    by 0x4024AD: main (cwdaemon.c:2249)
==80129== 
==80129== 1,664 bytes in 1 blocks are still reachable in loss record 16 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x55913FD: ??? (in /lib/libthr.so.3)
==80129==    by 0x558E28E: pthread_create (in /lib/libthr.so.3)
==80129==    by 0x4866259: cw_gen_start (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4034B4: cwdaemon_open_libcw_output (cwdaemon.c:709)
==80129==    by 0x403FCF: cwdaemon_handle_escaped_request (cwdaemon.c:1178)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== 4,992 bytes in 3 blocks are still reachable in loss record 18 of 18
==80129==    at 0x4851735: calloc (vg_replace_malloc.c:1599)
==80129==    by 0x55913FD: ??? (in /lib/libthr.so.3)
==80129==    by 0x558E28E: pthread_create (in /lib/libthr.so.3)
==80129==    by 0x4866259: cw_gen_start (in /home/acerion/root_dir/lib/libcw.so.8.0.0)
==80129==    by 0x4034B4: cwdaemon_open_libcw_output (cwdaemon.c:709)
==80129==    by 0x4035AD: cwdaemon_reset_libcw_output (cwdaemon.c:772)
==80129==    by 0x4035AD: cwdaemon_reset_almost_all (cwdaemon.c:666)
==80129==    by 0x403789: cwdaemon_handle_escaped_request (cwdaemon.c:953)
==80129==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==80129==    by 0x402547: main (cwdaemon.c:2288)
==80129== 
==80129== LEAK SUMMARY:
==80129==    definitely lost: 120 bytes in 6 blocks
==80129==    indirectly lost: 0 bytes in 0 blocks
==80129==      possibly lost: 0 bytes in 0 blocks
==80129==    still reachable: 10,595 bytes in 21 blocks
==80129==         suppressed: 4,096 bytes in 1 blocks
==80129== 
==80129== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0)
--80129-- 
--80129-- used_suppression:      1 MEMCHECK-LIBC-REACHABLE-1 /usr/local/libexec/valgrind/default.supp:582 suppressed: 4,096 bytes in 1 blocks
==80129== 
==80129== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0)

Result: PASS

There are some "definitely lost" records - they are caused by not making a
call to pthread_cond_destroy() in libcw. A TODO note will be added to libcw
to address this. I won't be fixing them during work on cwdaemon-0.13.0.

cwdaemon is calling cw_generator_stop() on exit, so those entries related to
cw_gen_start() aren't a problem of cwdaemon itself, but perhaps of libcw. I
won't be fixing them here, but I will add a TODO note in unixcw.

*** PASS functional tests

*Preconditions*

**** PASS unattended short tests

*Preconditions*

1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests

Run unattended tests, but without "long" variants. Run relatively short tests
and look at final result to have some indication of general status of tests
before delving deeper into testing procedures that require more attention.

*Steps*
1. Run "make check"
2. Let the unattended tests run and complete

*Expectations*

The unattended tests executed through "make check" pass. We can now perform
tests that require more attention from tester.

*Actual result*

PASS

Test of CWDEVICE Escape request requires another (second) cwdevice being
present. Without it it's not possible to find a non-default valid cwdevice.

**** PASS unattended long tests

Run unattended tests, but this time also with "long" variants. Run relatively
long tests and look at final result to have some indication of general status
of tests before delving deeper into testing procedures that require more
attention.

*Preconditions*

1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests

*Steps*
1. Run "make check"
2. Let the unattended tests run and complete

*Expectations*

The unattended tests executed through "make check" pass. We can now perform
tests that require more attention from tester.

*Actual result*

PASS

**** PASS PLAIN request

*Preconditions*

1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests
3. make && make check
4. Interrupt the build process when tests start being executed

*Test program*
./tests/functional_tests/unattended/request_plain/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*

PASS

**** PASS CARET request

*Preconditions*

1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests
3. make && make check
4. Interrupt the build process when tests start being executed

*Test program*
./tests/functional_tests/unattended/request_caret/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*
PASS

**** PASS Escape requests

***** PASS RESET        '0'

*Preconditions*

1. Connect TTY-to-USB converter.


*Info*

The following parameters are reset with RESET Escape request:
 - Morse speed,
 - tone (frequency) of played Morse code,
 - volume of played Morse code,
 - sound system used to play Morse code,
 - PTT delay
 - weighting of Morse code,
 - state of libcw,
 - word mode (to false),
 - state of pins of cwdevice,
 - cwdaemon's internal PTT flags


*Steps*
1. in terminal A start cwdaemon with specific values of some of parameters.
   Those will be the initial/default parameters.

   ./src/cwdaemon -n -x s --wpm 15 --tone 1000 --volume 100 --weighting 50

2. In terminal B start "nc" as a client of cwdaemon:

   nc -u 127.0.0.1 6789

3. In terminal B send some text to confirm that cwdaemon can play it with
   initial/default parameters.

4. In terminal B send few Escape requests to modify parameters of cwdaemon:
   - Morse speed: change to 8 (Esc 2)
   - tone: change to 400 (Esc 3)
   - volume: change to 20 (Esc g)
   - weighting: change to 0 (Esc 7)

5. In terminal B send some text to confirm that cwdaemon can play it with
   modified parameters.

6. In terminal B send few RESET Escape request to change cwdaemon's
   parameters back to their initial/default values.

7. In terminal B send some text to confirm that cwdaemon is playing the text
   with initial/default parameters.


*Expected result*
1. RESET Escape sent in step 6 does reset parameters to their initial values,
   which is confirmed by how cwdaemon plays a request in step 7.

*Actual result*
PASS

***** PASS SPEED        '2'

*Preconditions*

1. Connect TTY-to-USB converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc2.pl

*Steps*
1. In terminal B run a test
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors

*Actual result*
PASS

***** PASS TONE         '3'

*Preconditions*

1. Connect TTY-to-USB converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc3.pl

*Steps*
1. In terminal B run a test
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors

*Actual result*
PASS

***** PASS ABORT        '4'

*Preconditions*

1. Connect TTY-to-USB converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc64.pl

*Steps*
1. In terminal B run a test
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors
3. Text played by cwdaemon can be interrupted by ABORT Escape request when
   cwdaemon works in interruptible mode.
4. Text played by cwdaemon cannot be interrupted by ABORT Escape request when
   cwdaemon works in non-interruptible mode.
5. cwdaemon behaves as described in points 3 and 4 even when invalid ABORT
   Escape request is being sent, i.e. when the value of the request is
   non-empty.

*Actual result*
PASS

***** PASS EXIT         '5'

*Preconditions*
1. Connect TTY-to-USB converter.

*Test program*
./tests/functional_tests/unattended/request_esc_exit/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*
PASS

***** PASS WORD_MODE    '6'

*Preconditions*

1. Connect TTY-to-USB converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc64.pl

*Steps*
1. In terminal B run a test
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors
3. cwdaemon is being switched between word-mode (non-interruptible mode) and
   regular mode (interruptible mode).

   This is recognized by cwdaemon's response to ABORT Escape request.

   Played text cannot (in word-mode) or can (in regular mode) be interrupted
   by ABORT Escape request.

*Actual result*
PASS

***** PASS WEIGHTING    '7'

*Preconditions*

1. Connect TTY-to-USB converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc7.pl

*Steps*
1. In terminal B run a test
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. Weighting of dits and dashes changes as expected,
3. cwdaemon doesn't report any unexpected errors.

*Actual result*
PASS

***** PASS CWDEVICE     '8'

*Preconditions*
1. Attach two TTY-to-USB devices acting as two cwdevices: default one, and
   non-default one.

*Test program*
./tests/functional_tests/unattended/request_esc_cwdevice/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*
PASS

Low-priority ticket R0039 has been added to README.dev.org.

***** PASS PORT         '9'

*Preconditions*
1. Connect TTY-to-USB converter.

*Test program*
./tests/functional_tests/supervised/request_esc_port/test_program

*Steps*
1. Manually start the test
2. Read the banner text displayed at the beginning of test,
3. Observe the logs displayed during execution of test,
4. Wait for final status of the test.

*Expectations*
1. Final result of the test is a "PASS".
2. None of characteristics of played sounds are changing between iterations
   of test.

*Actual result*
PASS

***** PASS PTT_STATE    'a'

*Preconditions*

1. Prepare a multimeter or logic state analyzer.
2. Read information at the beginning of tests/cwtest_esca.pl script about 3
   modes.
3. Connect TTY-to-USB converter.
4. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
5. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esca.pl

*Steps*
1. Connect the multimeter to PTT pin of the TTY-to-USB converter.
2. In terminal B run a test

   perl ./<test program name> --mode X

   Do this with value of mode (X) equal to 0, 1, and 2.
3. Observe output of test in terminal B
4. Observe readings on the multimeter, showing state of PTT pin (ON or OFF).


*Expected result*
1. State of PTT pin changes at all, as shown by multimeter's reading,
2. State of PTT pin changes according to mode used by the test script.

*Actual result*
PASS

***** SKIPPED SSB_WAY      'b'

*Expected result*
Test passes

*Actual result*
SKIPPED: I don't have neither a HW needed to test this feature nor a test
scenario.

***** PASS TUNE         'c'
****** PASS Perl script
*Preconditions*

1. Connect TTY-to-USB converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escc.pl

*Steps*
1. In terminal B run a test
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors

*Actual result*
PASS

****** PASS Manual test, with observing PTT Flag

*Preconditions*

1. Connect TTY-to-USB converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s -iii -t 1
3. In another terminal (terminal B) run nc:
   nc -u 127.0.0.1 6789


*Steps*
1. In terminal B request long tuning time
   ^[c9
2. In terminal A observe that PTT pin is raised:

   [II] cwdaemon: -------------------
   [II] cwdaemon: received Escape request: "<ESC>c" / "<ESC>0x63"
   [II] cwdaemon: requested tuning time [s]: "9"
   [II] cwdaemon: PTT (TUNE) on
   [DD] cwdaemon: PTT flag +PTT_ACTIVE_AUTO (0x01/Ame)
   [DD] cwdaemon: keying event 1

3. After the tuning time elapses, observe in terminal A that PTT pin is
   lowered:

   [II] cwdaemon: low TQ callback: start, TQ len = 1, PTT flag = 0x01/Ame
   [II] cwdaemon: low TQ callback: branch 1, PTT flag = 0x01/Ame
   [DD] cwdaemon: PTT flag = 0 (0x00/ame)
   [II] cwdaemon: PTT (auto) off
   [II] cwdaemon: low TQ callback: end, TQ len = 1, PTT flag = 0x00/ame
   [DD] cwdaemon: keying event 0


*Expected result*
1. PTT pin is raised and lowered as expected,
2. cwdaemon doesn't report any unexpected errors


*Actual result*
PASS

***** PASS TX_DELAY     'd'

*Preconditions*

1. Connect TTY-to-USB converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escd.pl

*Steps*
1. In terminal B run a test
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors.

*Actual result*
PASS

***** SKIPPED BAND_SWITCH  'e'

*Expected result*
Test passes

*Actual result*
SKIPPED: I don't have neither a HW needed to test this feature nor a test
scenario.

***** PASS SOUND_SYSTEM 'f'

The test may be trying to switch to invalid sound system "?"/"<NONE>". Such
attempts will be ignored by cwdaemon, and cwdaemon SHOULD continue to use
previous sound system.

*Preconditions*
1. Attach TTY-to-USB device acting as cwdevice.

*Test program*
./tests/functional_tests/supervised/request_esc_sound_system/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*
PASS

***** PASS VOLUME       'g'

*Preconditions*

1. Connect TTY-to-USB converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escg.pl

*Steps*
1. In terminal B run a test
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. Volume of sounds produced by cwdaemon changes as expected,
3. cwdaemon doesn't report any unexpected errors.

*Actual result*
PASS

***** PASS REPLY        'h'

*Test program*
./tests/functional_tests/unattended/request_esc_reply/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*
PASS

**** PASS Command line options

***** PASS -h, --help

*Actual result*
PASS
Help text is printed, looks correct.

***** PASS -V, --version

*Actual result*
PASS
Information about version of cwdaemon and libcw is printed correctly.

***** PASS -d, --cwdevice <device>

*Preconditions*
1. Connect two cwdevices


*Steps*
1. In one terminal (terminal A) start a test instance of cwdaemon:

   ./src/cwdaemon -n -x s -t1 --cwdevice <cwdevice-1>

2. In another terminal (terminal B) send some text using nc. As the text is
   being played, confirm that cwdaemon is toggling "keying" pin on
   <cwdevice-1>, but not in <cwdevice-2>.

3. In terminal A stop cwdaemon and start it again, but this time with the
   second cwdevice.

   ./src/cwdaemon -n -x s -t1 -d <cwdevice-2>

4. In another terminal (terminal B) send some text using nc. As the text is
   being played, confirm that cwdaemon is toggling "keying" pin on
   <cwdevice-2>, but not in <cwdevice-1>.


*Expected result*
1. cwdaemon is using the right cwdevice, as specified in command line.


*Actual result*
PASS
cwdevice is using the device that was specified in command line.

***** PASS -o, --options <option>

*Test program*
./tests/functional_tests/unattended/option_cwdevice_tty_lines/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test
4. Manually run at least one test with the long "--options" form of the
   command line option.
5. Try passing invalid values of "key=" or "ptt=" keywords in command line.


*Expectations*
1. Final result of the test is a "PASS".
2. Handling of long "--options" option work.
3. Invalid values of "key=" or "ptt=" keywords are rejected.


*Actual result*
PASS

The test program doesn't support passing long form of option (--options) in
command line yet, so I had to do that part manually.

I also verified manually the handling of invalid values of "key=" and "ptt="
keywords.

***** PASS -n, --nofork

*Preconditions*

Find out the syslog output to which daemon which output a "daemon" processes
are logging.

On FreeBSD, per /etc/syslog.conf, the INFO messages go to
/var/log/daemon.log.


*Expected result*
1. cwdaemon doesn't daemonize when executed with "-n" flag.
2. cwdaemon does daemonize when executed without "-n" flag.
3. When cwdaemon doesn't daemonize, it doesn't log into syslog.
4. When cwdaemon does daemonize, it does log into syslog.


*Actual result*
PASS
1. cwdaemon obeys "-n" flag.
2. cwdaemon logs to proper output depending on "-n" flag.

***** PASS -p, --port <port>

*Test program*
./tests/functional_tests/unattended/option_port/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*
PASS

***** PASS -s, --wpm <speed>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*
PASS
Speed is set as requested in command line.
Values out of range are rejected.

***** PASS -t, --pttdelay <time>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon.
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*
PASS
Values in range are accepted.
Values out of range are rejected.

Unfortunately right now I don't have good tools or good SW tests to measure
the actual delay.

***** PASS -x, --system <sound system>

*Preconditions*
1. Connect a cwdevice


*Steps*
1. Run cwdaemon with different sound systems: one of 'n', 'c', 'o', 'a', 'p',
   's'.
2. Observe if cwdaemon exits due to unsupported sound system, or if it starts
   correctly.
3. If it starts correctly, try to sending some text through "nc".


*Expected results*
1. cwdaemon doesn't start correctly if a sound system not supported on the
   test platform is specified.
2. cwdaemon starts correctly if a sound system supported on the test platform
   is specified.
3. If cwdaemon starts correctly, it plays a text using specified sound
   system.
4. Both short and long form of the command line option works the same.


*Actual result*
PASS
1. cwdaemon accepted following sound systems: 'n', 'o', 's'.
2. Short and long form of command line option works the same.

***** PASS -v, --volume <volume>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*
PASS
Volume is set as requested in command line.
Values out of range are rejected.

***** PASS -w, --weighting <weight>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.
4. cwdaemon accepts negative values (you may need to put them in quotes)

*Actual result*
PASS
Tone is set as requested in command line.
Values out of range are rejected.

***** PASS -T, --tone <tone>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*
PASS.
Tone is set as requested in command line.
Values out of range are rejected.

***** PASS -i

*Preconditions*

1. Connect TTY-to-USB converter.

*Steps*

1. In one terminal (terminal A) start cwdaemon without "-i" flags:

   ./src/cwdaemon -n -x s -d <cwdevice>

2. In another terminal (terminal B) send a valid TONE Escape request, send an
   invalid TONE Escape request, and then play some text.

3. In terminal A observe that with no "-i" flags specified in command line,
   cwdaemon prints only errors and warnings.

4. In terminal A do a series of starts of cwdaemon with increasing amount of
   "-i" flags. In terminal B send a TONE Escape request, then play some text,
   and then in terminal A observe increasing verbosity of cwdaemon's logs.


*Expected result*
1. Verbosity of logs in cwdaemon increases with count of "i" options passed
   in command line.


*Actual result*
PASS

***** PASS -y, --verbosity <threshold>

*Preconditions*

1. Connect TTY-to-USB converter.

*Steps*

1. In one terminal (terminal A) start cwdaemon without "-y"/"--verbosity"
   flag:

   ./src/cwdaemon -n -x s -d <cwdevice>

2. In another terminal (terminal B) send a valid TONE Escape request, send an
   invalid TONE Escape request, and then play some text.

3. In terminal A observe that with no "-i" flags specified in command line,
   cwdaemon prints only errors and warnings.

4. In terminal A do a series of starts of cwdaemon with increasing a level
   passed to "-y"/"--verbosity" flag (n (none), e (error), w (warning), i
   (information), d (debug)). In terminal B send a TONE Escape request, then
   play some text, and then in terminal A observe increasing verbosity of
   cwdaemon's logs.

   In particular confirm that with "n" flag cwdaemon doesn't even print
   errors.


*Expected result*

1. Verbosity of logs in cwdaemon increases with level of verbosity passed to
   "-y"/"--verbosity" in command line.


*Actual result*
PASS

***** PASS -I, --libcwflags <flags>

Confirm that option that specifies debug flags for libcw is working.

In this test cwdaemon is executed:
 - in non-daemonized mode to observe console logs;
 - with null keying device because cwdevice is not essential for this test.


*Steps*

1. Open connection to cwdaemon.

   nc -u localhost 6789
   OR
   nc -u 127.0.0.1 6789

2. Run cwdaemon without the tested flag. Notice the 'd' (DEBUG) threshold for
   cwdaemon.

   ./src/cwdaemon -d null -n -x o -y d

3. Send some characters to cwdaemon with nc.

4. Confirm that cwdaemon's log output shows cwdaemon logs, but doesn't show
   any libcw logs (because '--libcwflags' option is not provided).

5. Kill cwdaemon started in step 2. Start cwdaemon with '--libcwflags'
   option. Notice the 'd' (DEBUG) threshold for cwdaemon.

   ./src/cwdaemon --libcwflags=4294967295 -d null  -n -x o -y d

6. Send characters to cwdaemon using nc.

7. Confirm that cwdaemon's log output shows cwdaemon logs and show libcw logs
   with severity DEBUG or higher.

8. Kill cwdaemon started in step 4. Start cwdaemon with '--libcwflags'
   option. Notice the 'w' (WARNING) threshold for cwdaemon.

   ./src/cwdaemon --libcwflags=4294967295 -d null  -n -x o -y w

9. Send characters to cwdaemon using nc.

10. Confirm that cwdaemon's log output shows cwdaemon logs. If there are any
    libcw logs, the logs have severity only WARNING or ERROR.

*Expected result*

1. libcw logs are visible in cwdaemon's log output only if `--libcwflags`
   flag is used

2. Threshold for libcw logs depends on value of '-y' flag passed to cwdaemon.

*Actual result*
PASS

***** PASS -f, --debugfile <output>

*Preconditions*

1. Connect TTY-to-USB converter.


*Steps*

1. Confirm that cwdaemon logs messages to text file when started like this in
   non-forking mode:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] /tmp/non_forking.txt

2. Confirm that cwdaemon logs messages to text file when started like this in
   forking mode:

   ./src/cwdaemon -x s -d /dev/cuaU0 -y d [-f/--debugfile] /tmp/forking.txt

3. Confirm that cwdaemon logs messages to stdout when started like this:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] stdout 2>/tmp/stderr.txt

4. Confirm that cwdaemon logs messages to stderr when started like this:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] stderr 1>/tmp/stdout.txt

5. Confirm that cwdaemon rejects "syslog" as output of logs when started like
   this:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] syslog


*Expected result*
1. All of the behaviours from "Steps" are seen in running cwdaemon.
2. All of the behaviours from "Steps" are the same for short and long form of
   the command line option.


*Actual result*
PASS

**** PASS Misc tests

***** PASS reset register callback

*Preconditions*

1. Connect TTY-to-USB converter.
2. Compile cwdaemon with functional tests enabled

   ./configure --enable-functional-tests
   make
   make check


*Test program*
./tests/functional_tests/unattended/reset_register_callback/test_program


*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test


*Expectations*
Final result of the test is a "PASS"


*Actual result*
PASS

***** PASS State of tty DTR and RTS pins during inactivity

*Purpose*

Confirm that cwdaemon doesn't change initial state of tty cwdevice's DTR and
RTS pins right after start.

Incorrect initialization of tty device has led to
https://github.com/acerion/cwdaemon/issues/12.

You can use cwdaemon 0.10.2 as a baseline. In a version following 0.10.2 a
change in ttys_init() was made that introduced an undesired behaviour of the
pins.


*Prerequisites*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. A device capable of showing a logical state on converter's pins: a
   multimeter or logical state analyzer.


*Steps*

1. Connect USB-to-UART converter to USB port.

2. Measure initial state of DTR and RTS pins

   Expectation: the state should be "high".

3. Start cwdaemon, specify a proper cwdevice through "-d" option

   ./src/cwdaemon -n -x s -ii -d cuaU0

4. Measure again the state of DTR and RTS pins.

   Expectation: cwdaemon should not change the state of the pins during start
   of cwdaemon and initialization of the cwdevice. The state on both pins
   should be "high".

5. Send some text to be played by cwdaemon

   nc -u 127.0.0.1 6789
   <Enter some text>

   After text is played and keyed on cwdevice, measure state of DTR and RTS
   pins.

   Expectation: after the text is played and keyed on cwdevice, the state of
   both pins should be "high".

6. Send "reset" command to cwdaemon

   Send "<ESC>0" Escape request to cwdaemon.

   nc -u 127.0.0.1 6789
   <Press ESC key>
   <Press "0" key>
   <Press Enter key>


   In cwdaemon's logs look for confirmation that reset is being performed:

   [INFO ] cwdaemon: requested resetting of parameters
   [INFO ] cwdaemon: resetting completed


   Wait for 2 seconds for the end of reset, then measure state of DTR and RTS
   pins.

   Expectation: after a reset is completed, the state of both pins should be
   "high".

7. Terminate cwdaemon

   Either press Ctrl-C while cwdaemon is running in foreground, or send
   "<ESC>5" Escape request to cwdaemon.

   Then measure state of DTR and RTS pins.

   Expectation: after an exit is performed, the state of both pins should be
   "high".


*Expected result*

All expectations from above steps are met.

*Actual result*
PASS

***** PASS tests/cwtest.pl

*Preconditions*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. cwdaemon must be started (automatically or manually), running on default
   port, with <platform's preferred sound system>.

   ./src/cwdaemon -n -x s -d cuaU0


*Test program*
cwtest.pl


*Steps*
1. Enter "tests" directory
2. Start the test script

   perl ./cwtest.pl --cycles 2

3. Observe test's output and cwdaemon's output.


*Expected result*
1. There are no errors in messages printed by cwdaemon.
2. There are no errors in messages printed by the test.


*Actual result*
PASS

***** PASS tests/cwtest_short_space.pl

*Preconditions*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. cwdaemon must be started (automatically or manually), running on default
   port, with <platform's preferred sound system>.

   ./src/cwdaemon -n -x s -d cuaU0


*Test program*
cwtest_short_space.pl


*Steps*
1. Enter "tests" directory
2. Start the test script

   perl ./cwtest_short_space.pl --cycles 20

3. Observe test's output and cwdaemon's output.


*Expected result*
1. There are no errors in messages printed by cwdaemon.
2. There are no errors in messages printed by the test.


*Actual result*
PASS

*** FAILED gcov/lcov

*Purpose*

Confirm that it's possible to generate a coverage report for unit tests. It
is not expected that specific level of coverage is achieved.


*Prerequisites*

1. "lcov" package is installed.
2. The compilation and the coverage must be done with the same toolchain, and
   therefore on FreeBSD you may need to do this:

   export CC=clang


*Steps*

The steps are copied from relevant sections in top-level README file:

1. ./configure --enable-functional-tests --enable-gcov
2. make gcov
3. <web browser> tests/coverage/unit_tests/daemon_options/index.html
4. make clean


*Expected results*

1. Summary of configuration step correctly shows enabled gcov,
2. There are no errors during generation of gcov/lcov report,
3. Contents of the report looks correct (no obvious problems with the
   report).
4. There are no errors during cleanup step.


*Actual result*
FAIL

I can't get gcov on FreeBSD work quickly.

Given that gcov coverage reports are not a key feature of cwdaemon, and given
that it works on Linux, I will let it FAIL today on FreeBSD.

There is a ticket R0045 that addresses this problem.

*** PASS memory tests with valgrind
**** PASS memory tests with valgrind for functional/unattended tests

*Goal*

Run all of cwdaemon's unattended functional tests in a setup where cwdaemon is
running under control of valgrind.


*Steps*

1. Enable compilation of supervised functional tests

   ./configure --enable-functional-tests --enable-long-functional-tests

2. Compile cwdaemon

   make && make check

   You can terminate the "make check" command as soon as build system starts
   executing unit tests.

3. Prepare environment variables that will be used by all test programs
   executed with "make check"

   export CWDAEMON_TEST_SOUND_SYSTEM=null
   export CWDAEMON_TEST_SUPERVISOR=valgrind

4. Find all unattended tests

   find tests/functional_tests/unattended/ -name test_program

5. Run each binary by hand, one at a time.

   The purpose of running each test at a time is to get informed about a
   failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

tests/functional_tests/unattended/option_cwdevice_tty_lines/test_program - ticket R0040
tests/functional_tests/unattended/request_esc_exit/test_program          - PASS
tests/functional_tests/unattended/reset_register_callback/test_program   - PASS
tests/functional_tests/unattended/option_port/test_program               - PASS, ticket R0041
tests/functional_tests/unattended/request_plain/test_program             - PASS
tests/functional_tests/unattended/request_caret/test_program             - PASS
tests/functional_tests/unattended/request_esc_cwdevice/test_program      - PASS
tests/functional_tests/unattended/request_esc_reply/test_program         - PASS



Tests of request_esc_cwdevice result in this valgrind log (and similar logs);

==86935== 2 bytes in 1 blocks are still reachable in loss record 1 of 9
==86935==    at 0x484D314: malloc (vg_replace_malloc.c:450)
==86935==    by 0x534555C: strdup (in /lib/libc.so.7)
==86935==    by 0x5273152: ??? (in /lib/libc.so.7)
==86935==    by 0x53455C6: ??? (in /lib/libc.so.7)
==86935==    by 0x53457C0: strerror (in /lib/libc.so.7)
==86935==    by 0x405472: tty_probe_cwdevice (ttys.c:112)
==86935==    by 0x40449B: cwdaemon_cwdevice_set (cwdaemon.c:2420)
==86935==    by 0x405BBA: cwdaemon_option_cwdevice (options.c:148)
==86935==    by 0x403A53: cwdaemon_handle_escaped_request (cwdaemon.c:1049)
==86935==    by 0x4043E0: cwdaemon_receive (cwdaemon.c:921)
==86935==    by 0x402547: main (cwdaemon.c:2297)

Since strerror() returns pointer to static buffer that can't be deallocated,
I'm not going to address this problem.





Tests of option_port result in these two valgrind issues:

==10394== FILE DESCRIPTORS: 4 open (3 std) at exit.
==10394== Open AF_UNIX socket 3: <unknown>
==10394==    at 0x52B898A: socket (in /lib/libc.so.7)
==10394==    by 0x5237C5A: ??? (in /lib/libc.so.7)
==10394==    by 0x52377B0: ??? (in /lib/libc.so.7)
==10394==    by 0x5237205: syslog (in /lib/libc.so.7)
==10394==    by 0x404848: log_message (log.c:140)
==10394==    by 0x40592E: cwdaemon_option_network_port (options.c:51)
==10394==    by 0x402123: cwdaemon_args_process_short (cwdaemon.c:1664)
==10394==    by 0x402123: cwdaemon_args_process_long (cwdaemon.c:1634)
==10394==    by 0x402123: cwdaemon_args_parse (cwdaemon.c:2119)
==10394==    by 0x402123: main (cwdaemon.c:2165)
==10394== 
==10394== 
==10394== HEAP SUMMARY:
==10394==     in use at exit: 25,168 bytes in 3 blocks
==10394==   total heap usage: 8 allocs, 5 frees, 104,616 bytes allocated
==10394== 
==10394== 23,440 bytes in 1 blocks are still reachable in loss record 3 of 3
==10394==    at 0x484D314: malloc (vg_replace_malloc.c:450)
==10394==    by 0x52AFCB0: ??? (in /lib/libc.so.7)
==10394==    by 0x52AFF75: ??? (in /lib/libc.so.7)
==10394==    by 0x52373FA: ??? (in /lib/libc.so.7)
==10394==    by 0x5237205: syslog (in /lib/libc.so.7)
==10394==    by 0x404848: log_message (log.c:140)
==10394==    by 0x40592E: cwdaemon_option_network_port (options.c:51)
==10394==    by 0x402123: cwdaemon_args_process_short (cwdaemon.c:1664)
==10394==    by 0x402123: cwdaemon_args_process_long (cwdaemon.c:1634)
==10394==    by 0x402123: cwdaemon_args_parse (cwdaemon.c:2119)
==10394==    by 0x402123: main (cwdaemon.c:2165)

The first one is covered by new ticket R0041, and it doesn't appear to be a
serious issue for now.

The second one doesn't seem to be a real issue, so I will ignore it.

**** PASS memory tests with valgrind for functional/supervised tests

*Goal*

Run all of cwdaemon's supervised functional tests in a setup where cwdaemon
is running under control of valgrind.


*Steps*

1. Enable compilation of supervised functional tests

   ./configure --enable-functional-tests --enable-long-functional-tests

2. Compile cwdaemon

   make && make check

   You can terminate the "make check" command as soon as build system starts
   executing unit tests.

3. Prepare environment variables that will be used by all test programs
   executed with "make check"

   export CWDAEMON_TEST_SOUND_SYSTEM=null
   export CWDAEMON_TEST_SUPERVISOR=valgrind

4. Find all supervised tests

   find tests/functional_tests/supervised/ -name test_program

5. Run each binary by hand, one at a time.

   The purpose of running each test at a time is to get informed about a
   failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

tests/functional_tests/supervised/feature_multiple_requests/test_program   - SKIPPED (this test is not fully ready yet)
tests/functional_tests/supervised/request_esc_sound_system/test_program    - PASS (problem in libcw)
tests/functional_tests/supervised/request_esc_port/test_program            - PASS


Tests of request_esc_sound_system reported a known problem in libcw/tq:

==4930== 20 bytes in 1 blocks are definitely lost in loss record 1 of 10
==4930==    at 0x4852105: calloc (vg_replace_malloc.c:1679)
==4930==    by 0x559467D: pthread_cond_init (in /lib/libthr.so.3)
==4930==    by 0x486E4C0: cw_tq_new_internal (libcw_tq.c:185)
==4930==    by 0x4866F76: cw_gen_new (libcw_gen.c:498)
==4930==    by 0x4865B00: cw_generator_new_internal (libcw.c:182)
==4930==    by 0x4865AE6: cw_generator_new (libcw.c:158)
==4930==    by 0x403426: cwdaemon_open_libcw_output (cwdaemon.c:693)
==4930==    by 0x4035AD: cwdaemon_reset_libcw_output (cwdaemon.c:772)
==4930==    by 0x4035AD: cwdaemon_reset_almost_all (cwdaemon.c:666)
==4930==    by 0x4024AD: main (cwdaemon.c:2257)
==4930== 
==4930== 140 bytes in 7 blocks are definitely lost in loss record 5 of 10
==4930==    at 0x4852105: calloc (vg_replace_malloc.c:1679)
==4930==    by 0x559467D: pthread_cond_init (in /lib/libthr.so.3)
==4930==    by 0x486E4C0: cw_tq_new_internal (libcw_tq.c:185)
==4930==    by 0x4866F76: cw_gen_new (libcw_gen.c:498)
==4930==    by 0x4865B00: cw_generator_new_internal (libcw.c:182)
==4930==    by 0x4865AE6: cw_generator_new (libcw.c:158)
==4930==    by 0x403426: cwdaemon_open_libcw_output (cwdaemon.c:693)
==4930==    by 0x403CA1: cwdaemon_handle_escaped_request (cwdaemon.c:1173)
==4930==    by 0x404360: cwdaemon_receive (cwdaemon.c:920)
==4930==    by 0x402547: main (cwdaemon.c:2296)

This problem can't be fixed in cwdaemon, so I'm ignoring it here.

**** PASS memory tests with valgrind for fuzzing tests

*Goal*

Run all of cwdaemon's fuzzing tests in a setup where cwdaemon is running
under control of valgrind.


*Steps*

1. Confirm that fuzzing tests done few sections above were already executed
   with cwdaemon being under control of valgrind.

*Expected result*

No additional test is needed here because the primary fuzzing tests already
use valgrind.

*Actual result*
PASS

**** PASS memory tests with valgrind for unit tests

*Goal*

Run all of cwdaemon's unit tests in a setup where unit tests are running
under control of valgrind.


*Steps*

1. Enable compilation of unit tests (unit tests of cwdaemon only)

   make clean
   ./configure

2. Compile cwdaemon and unit tests

   make && make check

3. Find all unit test binaries

   find tests/unit_tests/* -type f  -executable

4. Run each binary by hand, one at a time.

   /usr/local/bin/valgrind -s --leak-check=full --show-leak-kinds=all --track-fds=yes <path to test binary>

   The purpose of running each test at a time is to get informed about a
   failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

tests/unit_tests/daemon_options    - PASS
tests/unit_tests/daemon_sleep      - PASS
tests/unit_tests/daemon_utils      - PASS

** PASS Debian GNU/Linux 12
<Additional info about machine, OS, libs, compiler>
OS: Debian GNU/Linux 12
C compiler: gcc (Debian 12.2.0-14) 12.2.0
libc: glibc
unixcw/libcw: 3.6.1/8:0:0

*** PASS tests of build system
**** PASS make distcheck
*Steps*

Run the steps in main directory of project.

On FreeBSD you may need to use "gmake" (GNU make) instead of FreeBSD's
default "make".

1. ./configure
2. make distcheck

*Expected result*
1. Distribution package is built correctly (no errors during creation of
   archive).
2. Automatic tests executed during 'make distcheck' pass.
3. The distribution package contains all files intended for distribution (use
   your best judgement here).

*Actual result*
PASS

**** PASS ./configure flags

Test that all flags supported by ./configure script produce "buildable" code.

*Steps*
1. Run "make clean && ./configure && make && make check",
2. Run "make clean && ./configure --enable-functional-tests && make && make check",
3. Run "make clean && ./configure --enable-functional-tests --enable-long-functional-tests && make && make check",

*Expected result*

Each of the above tests results in correct build: binaries are built
correctly, including binaries for all kinds of tests.

You don't have to do full tests with these test binaries, just check that
they are there.

*Actual result*
PASS

**** PASS regression with make distcheck

Confirm that failure of 'make distcheck' after specific sequence of steps does not occur.

On Alpine 3.17 and with certain version of tests/unit_tests/Makefile.am, the
steps ended with compilation error, and distribution package was not created.

The error message was:

gcc  -g -O2 -Wall -Wextra -pedantic -Wswitch-default -Wimplicit-function-declaration -Wswitch-enum -Wfloat-equal -Wpointer-arith -Wcast-qual -Wwrite-strings -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wredundant-decls -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict -Wnull-dereference -Wjump-misses-init -Wdouble-promotion -Wshadow -Wformat=2 -std=c99   -o utils ../../src/utils-utils.o utils-utils.o  
/usr/lib/gcc/x86_64-alpine-linux-musl/12.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find utils-utils.o: No such file or directory
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:328: utils] Error 1
make[4]: Leaving directory '/home/acerion/cwdaemon/git_repo/cwdaemon-0.12.0/_build/sub/tests/unit_tests'



*Steps*
make distclean
autoreconf
automake
./configure
make
make check
make distcheck  # <---- In this step a compilation error was occurring


*Expected result*
Distribution archive is created correctly.

*Actual result*
PASS

*** PASS unit tests

Execute tests implemented in tests/unit_tests.

*Steps*
1. ./configure --enable-functional-tests
2. make check
3. Wait for all unit tests to be executed. You can terminate the tests as
   soon as unit tests complete, and functional tests start being executed.

*Expected result*
All unit tests should have "PASS" status in output of "make check" command.

This shows examples of logs for unit tests (notice the "unit_tests"
directory):
PASS: unit_tests/daemon_utils
PASS: unit_tests/daemon_options
PASS: unit_tests/tests_random
PASS: unit_tests/tests_time_utils

*Notes*

Count of unit tests depends on whether you pass "--enable-functional-tests"
to ./configure or not. Test both cases.

*Actual result*

 - With functional tests not enabled: PASS

 - With functional tests enabled: PASS

*** PASS fuzzing tests
**** PASS simple fuzzing test

*Purpose*

Run simple fuzzing test to catch memory errors.

Run the test early in testing phase, certainly before doing manual tests. Let
an unattended fuzzing test detect problems early.


*Preconditions*

1. Be sure to use large count of iterations in the test: set value of
   "n_iters" in test_run() to high value.

2. Be sure to run the test under valgrind: set "supervisor_id" to
   "supervisor_id_valgrind" in options passed to server_start().

3. Be sure to have all tests enabled in "g_test_cases[]".

4. Plug-in an USB-to-UART converter.


*Steps*

1. Compile cwdaemon package with fuzzing test enabled:

   ./configure --enable-functional-tests
   make
   make check

   You can interrupt the automatic execution of unit and functional tests.

2. Run test program: tests/fuzzing/simple/test_program
3. Look at summary of valgrind test, looking for information about memory
   errors.


*Expected result*

1. valgrind reports no memory leaks (with possible exception of memory leak
   described in ticket R0018).

2. valgrind reports no memory errors of other types.

3. cwdaemon's test framework doesn't report any errors on its own

*Actual result*
PASS

*** PASS functional tests

**** PASS unattended short tests

*Goal*

Run unattended tests, but without "long" variants. Run relatively short tests
and look at final result to have some indication of general status of tests
before delving deeper into testing procedures that require more attention.


*Preconditions*

1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests
3. make


*Steps*

1. Run "make check"
2. Let the unattended tests run and complete


*Expectations*

The unattended tests executed through "make check" pass. We can now perform
tests that require more attention from tester.


*Actual result*
PASS

**** PASS unattended long tests

*Goal*

Run unattended tests, but this time also with "long" variants. Run relatively
long tests and look at final result to have some indication of general status
of tests before delving deeper into testing procedures that require more
attention.


*Preconditions*

1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests
3. make


*Steps*

1. Run "make check"
2. Let the unattended tests run and complete


*Expectations*

The unattended tests executed through "make check" pass. We can now perform
tests that require more attention from tester.


*Actual result*
PASS

**** PASS PLAIN request

*Preconditions*
1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests
3. make && make check
4. Interrupt the build process when tests start being executed


*Test program*
./tests/functional_tests/unattended/request_plain/test_program


*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test


*Expectations*
Final result of the test is a "PASS"


*Actual result*
PASS

**** PASS CARET request

*Preconditions*
1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests
3. make && make check
4. Interrupt the build process when tests start being executed


*Test program*
tests/functional_tests/unattended/request_caret/test_program


*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test


*Expectations*
Final result of the test is a "PASS"


*Actual result*
PASS

**** PASS Escape requests

***** PASS RESET        '0'

*Preconditions*

1. Connect USB-to-UART converter.


*Info*

The following parameters are reset with RESET Escape request:
 - Morse speed,
 - tone (frequency) of played Morse code,
 - volume of played Morse code,
 - sound system used to play Morse code,
 - PTT delay
 - weighting of Morse code,
 - state of libcw,
 - word mode (to false),
 - state of pins of cwdevice,
 - cwdaemon's internal PTT flags


*Steps*
1. in terminal A start cwdaemon with specific values of some of parameters.
   Those will be the initial/default parameters.

   ./src/cwdaemon -n -x s --wpm 15 --tone 1000 --volume 100 --weighting 50

2. In terminal B start "nc" as a client of cwdaemon:

   nc -u 127.0.0.1 6789

3. In terminal B send some text to confirm that cwdaemon can play it with
   initial/default parameters.

4. In terminal B send few Escape requests to modify parameters of cwdaemon:
   - Morse speed: change to 8 (Esc 2)
   - tone: change to 400 (Esc 3)
   - volume: change to 20 (Esc g)
   - weighting: change to 0 (Esc 7)

5. In terminal B send some text to confirm that cwdaemon can play it with
   modified parameters.

6. In terminal B send RESET Escape request to change cwdaemon's parameters
   back to their initial/default values.

7. In terminal B send some text to confirm that cwdaemon is playing the text
   with initial/default parameters.


*Expected result*
1. RESET Escape sent in step 6 does reset parameters to their initial values,
   which is confirmed by how cwdaemon plays a request in step 7.

*Actual result*
PASS

***** PASS SPEED        '2'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc2.pl

*Steps*
1. In terminal B run a test
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors

*Actual result*
PASS

***** PASS TONE         '3'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc3.pl

*Steps*
1. In terminal B run the test program
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors

*Actual result*
PASS

***** PASS ABORT        '4'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc64.pl

*Steps*
1. In terminal B run the test program
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors
3. Text played by cwdaemon can be interrupted by ABORT Escape request when
   cwdaemon works in interruptible mode.
4. Text played by cwdaemon cannot be interrupted by ABORT Escape request when
   cwdaemon works in non-interruptible mode.
5. cwdaemon behaves as described in points 3 and 4 even when invalid ABORT
   Escape request is being sent, i.e. when the value of the request is
   non-empty.

*Actual result*
PASS

***** PASS EXIT         '5'

*Preconditions*
1. Connect USB-to-UART converter.

*Test program*
./tests/functional_tests/unattended/request_esc_exit/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*
PASS

***** PASS WORD_MODE    '6'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc64.pl

*Steps*
1. In terminal B run the test program
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors
3. cwdaemon is being switched between word-mode (non-interruptible mode) and
   regular mode (interruptible mode).

   A current mode can be recognized by cwdaemon's response to ABORT Escape
   request.

   Played text cannot (in word-mode) or can (in regular mode) be interrupted
   by ABORT Escape request.

*Actual result*
PASS

***** PASS WEIGHTING    '7'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc7.pl

*Steps*
1. In terminal B run the test program
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. Weighting of dits and dashes changes as expected,
3. cwdaemon doesn't report any unexpected errors.

*Actual result*
PASS

***** PASS CWDEVICE     '8'

*Preconditions*
1. Attach two USB-to-UART devices acting as two cwdevices: default one, and
   non-default one.

*Test program*
./tests/functional_tests/unattended/request_esc_cwdevice/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*
PASS

***** PASS PORT         '9'

*Preconditions*
1. Connect USB-to-UART converter.

*Test program*
./tests/functional_tests/supervised/request_esc_port/test_program

*Steps*
1. Manually start the test
2. Read the banner text displayed at the beginning of test,
3. Observe the logs displayed during execution of test,
4. Wait for final status of the test.

*Expectations*
1. Final result of the test is a "PASS".
2. None of characteristics of played sounds are changing between iterations
   of test.
3. You can see the following message in logs of cwdaemon when PORT Escape
   request is sent to cwdaemon:

   [WW] cwdaemon: obsolete request "9" (change network port), ignoring

*Actual result*
PASS

***** PASS PTT_STATE    'a'

*Preconditions*

1. Prepare a multimeter or logic state analyzer.
2. Read information at the beginning of tests/cwtest_esca.pl script about 3
   modes.
3. Connect USB-to-UART converter.
4. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s --cwdevice <cwdevice>
5. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esca.pl

*Steps*
1. Connect the multimeter to PTT pin of the USB-to-UART converter.
2. In terminal B run the test program

   perl ./<test program name> --mode X

   Do this with value of mode (X) equal to 0, 1, and 2.
3. Observe output of test in terminal B
4. Observe readings on the multimeter, showing state of PTT pin (ON or OFF).


*Expected result*
1. State of PTT pin changes at all, as shown by multimeter's reading,
2. State of PTT pin changes according to mode used by the test script.

*Actual result*
PASS

***** SKIPPED SSB_WAY      'b'

SKIPPED: I don't have neither a HW needed to test this feature nor a test
scenario.

***** PASS TUNE         'c'
****** PASS Perl script

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escc.pl

*Steps*
1. In terminal B run the test program
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B
4. Count the duration of each generated tuning tone.


*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors,
3. Each generated tuning tone lasted for expected count of seconds.


*Actual result*
PASS

****** PASS Manual test, with observing PTT Flag

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s -iii -t 1
3. In another terminal (terminal B) run nc:
   nc -u 127.0.0.1 6789


*Steps*
1. In terminal B request long tuning time
   ^[c9
2. In terminal A observe that PTT pin is raised:

   [II] cwdaemon: -------------------
   [II] cwdaemon: received Escape request: "<ESC>c" / "<ESC>0x63"
   [II] cwdaemon: requested tuning time [s]: "9"
   [II] cwdaemon: PTT (TUNE) on
   [DD] cwdaemon: PTT flag +PTT_ACTIVE_AUTO (0x01/Ame)
   [DD] cwdaemon: keying event 1

3. After the tuning time elapses, observe in terminal A that PTT pin is
   lowered:

   [II] cwdaemon: low TQ callback: start, TQ len = 1, PTT flag = 0x01/Ame
   [II] cwdaemon: low TQ callback: branch 1, PTT flag = 0x01/Ame
   [DD] cwdaemon: PTT flag = 0 (0x00/ame)
   [II] cwdaemon: PTT (auto) off
   [II] cwdaemon: low TQ callback: end, TQ len = 1, PTT flag = 0x00/ame
   [DD] cwdaemon: keying event 0


*Expected result*
1. PTT pin is raised and lowered as expected,
2. cwdaemon doesn't report any unexpected errors


*Actual result*
PASS

***** PASS TX_DELAY     'd'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escd.pl

*Steps*
1. In terminal B run the test program
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors.

*Actual result*
PASS

***** SKIPPED BAND_SWITCH  'e'

*Expected result*
Test passes


*Actual result*
SKIPPED: I don't have neither a HW needed to test this feature nor a test
scenario.

***** PASS SOUND_SYSTEM 'f'

The test may be trying to switch to invalid sound system "?"/"<NONE>". Such
attempts will be ignored by cwdaemon, and cwdaemon SHOULD continue to use
previous sound system.

*Preconditions*
1. Attach USB-to-UART device acting as cwdevice.

*Test program*
./tests/functional_tests/supervised/request_esc_sound_system/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*
PASS

***** PASS VOLUME       'g'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escg.pl

*Steps*
1. In terminal B run the test program
   perl ./<test program name>
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. Volume of sounds produced by cwdaemon changes as expected,
3. cwdaemon doesn't report any unexpected errors.

*Actual result*

***** PASS REPLY        'h'

*Test program*
./tests/functional_tests/unattended/request_esc_reply/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*
PASS

**** PASS Command line options

***** PASS -h, --help

***** PASS -V, --version

***** PASS -d, --cwdevice <device>

*Preconditions*
1. Connect two cwdevices


*Steps*
1. In one terminal (terminal A) start a test instance of cwdaemon:

   ./src/cwdaemon -n -x s --cwdevice <cwdevice-1>

2. In another terminal (terminal B) send some text using nc. As the text is
   being played, confirm that cwdaemon is toggling "keying" pin on
   <cwdevice-1>, but not in <cwdevice-2>.

3. In terminal A stop cwdaemon and start it again, but this time with the
   second cwdevice.

   ./src/cwdaemon -n -x s --cwdevice <cwdevice-2>

4. In another terminal (terminal B) send some text using nc. As the text is
   being played, confirm that cwdaemon is toggling "keying" pin on
   <cwdevice-2>, but not in <cwdevice-1>.


*Expected result*
1. cwdaemon is using the right cwdevice, as specified in command line.


*Actual result*
PASS

***** PASS -o, --options <option>

*Test program*
./tests/functional_tests/unattended/option_cwdevice_tty_lines/test_program

*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test
4. Manually run at least one test with the long "--options" form of the
   command line option.
5. Try passing invalid values of "key=" or "ptt=" keywords in command line.


*Expectations*
1. Final result of the test is a "PASS".
2. Handling of long "--options" option work.
3. Invalid values of "key=" or "ptt=" keywords are rejected.


*Actual result*
PASS

***** PASS -n, --nofork

*Preconditions*

Find out the syslog output to which daemon which output a "daemon" processes
are logging.

On FreeBSD, per /etc/syslog.conf, the INFO messages go to
/var/log/daemon.log.

On Linux, you will be able to see messages in output of "journalctl -f":
<netkeyer[5093]: received request: "is this syslog?">

****** PASS do not fork

*Steps*
1. Run cwdaemon in non-forking mode

   ./src/cwdaemon [-n/--nofork] -x p -ii

2. In another terminal send some PLAIN message to cwdaemon.


*Expected result*
1. Observe that cwdaemon is still attached to current terminal (it didn't go
   to background).
2. Observe that cwdaemon is logging messages to stdout


*Actual result*
PASS

****** PASS do fork

*Steps*

1. Confirm that no other cwdaemon is already running

   ps -A | grep cwdaemon

2. Run cwdaemon in forking mode

   ./src/cwdaemon -x p -ii

3. Confirm that cwdaemon process is present with this command:

   ps -A | grep cwdaemon

4. In another terminal send some PLAIN message to cwdaemon.

5. In another terminal send EXIT Escape request (<ESC>5).

6. Confirm that cwdaemon process is no longer present (because it exited)
   with this command:

   ps -A | grep cwdaemon


*Expected result*

1. After step 2 the cwdaemon is no longer attached to current terminal (it
   went to background).
2. In step 4 cwdaemon is not logging messages to stdout.
3. In step 4 cwdaemon is logging messages to syslog instead.
4. In step 4 cwdaemon is playing Morse code using sound system.


*Actual result*

PASS

***** PASS -p, --port <port>

*Test program*
./tests/functional_tests/unattended/option_port/test_program


*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test


*Expectations*
Final result of the test is a "PASS"


*Actual result*
PASS

***** PASS -s, --wpm <speed>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*
PASS

***** PASS -t, --pttdelay <time>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon.
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*
PASS

Unfortunately right now I don't have good tools or good SW tests to measure
the actual delay.

***** PASS -x, --system <sound system>

*Preconditions*
1. Connect a cwdevice


*Steps*
1. Run cwdaemon with different sound systems: one of 'n', 'c', 'o', 'a', 'p',
   's'.
2. Run cwdaemon with invalid sound system, e.g. 'u'.
3. Observe if cwdaemon exits due to unsupported sound system, or if it starts
   correctly.
4. If it starts correctly, try to sending some text through "nc".


*Expected results*
1. cwdaemon doesn't start correctly if a sound system not supported on the
   test platform is specified.
2. cwdaemon starts correctly if a sound system supported on the test platform
   is specified.
3. If cwdaemon starts correctly, it plays a text using specified sound
   system.
4. If cwdaemon starts correctly, Morse code is keyed on cwdevice's 'key' pin.
5. Both short and long form of the command line option works the same.


*Actual result*
PASS

***** PASS -v, --volume <volume>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*
PASS

***** PASS -w, --weighting <weight>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.
4. cwdaemon accepts negative values (you may need to put them in quotes)

*Actual result*

***** PASS -T, --tone <tone>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*
PASS

***** PASS -i

*Preconditions*

1. Connect USB-to-UART converter.

*Steps*

1. In one terminal (terminal A) start cwdaemon without "-i" flags:

   ./src/cwdaemon -n -x s -d <cwdevice>

2. In another terminal (terminal B) send a valid TONE Escape request, send an
   invalid TONE Escape request, and then play some text.

3. In terminal A observe that with no "-i" flags specified in command line,
   cwdaemon prints only errors and warnings

4. In terminal A do a series of starts of cwdaemon with increasing amount of
   "-i" flags. In terminal B send a TONE Escape request, then play some text,
   and then in terminal A observe increasing verbosity of cwdaemon's logs.


*Expected result*
1. Verbosity of logs in cwdaemon increases with count of "i" options passed
   in command line.


*Actual result*
PASS

***** PASS -y, --verbosity <threshold>

*Preconditions*

1. Connect USB-to-UART converter.

*Steps*

1. In one terminal (terminal A) start cwdaemon without "-y"/"--verbosity"
   flag:

   ./src/cwdaemon -n -x s -d <cwdevice>

2. In another terminal (terminal B) send a valid TONE Escape request
   (<ESC>3), send an invalid TONE Escape request, and then play some text.

3. In terminal A observe that with no "-y"/"--verbosity" flag specified in
   command line, cwdaemon prints only errors and warnings.

4. In terminal A do a series of starts of cwdaemon with increasing a
   verbosity level passed to "-y"/"--verbosity" flag (n (none), e (error), w
   (warning), i (information), d (debug)). In terminal B send a TONE Escape
   request, then play some text, and then in terminal A observe increasing
   verbosity of cwdaemon's logs.

   In particular confirm that with "n" flag cwdaemon doesn't even print
   errors.


*Expected result*

1. Verbosity of logs in cwdaemon increases with level of verbosity passed to
   "-y"/"--verbosity" in command line.


*Actual result*
PASS

***** PASS -I, --libcwflags <flags>

*Goal*
Confirm that option that specifies debug flags for libcw is working.


*Info*
In this test cwdaemon is executed:
 - in non-daemonized mode to observe console logs;
 - with null keying device because cwdevice is not essential for this test.


*Steps*

1. In terminal A, open connection to cwdaemon.

   nc -u localhost 6789
   OR
   nc -u 127.0.0.1 6789

2. In terminal B, run cwdaemon without the tested flag. Notice the 'd'
   (DEBUG) threshold for cwdaemon.

   ./src/cwdaemon -d null -n -x s -y d

3. In terminal A, send some characters to cwdaemon with nc.

4. Confirm that cwdaemon's log output shows cwdaemon logs, but doesn't show
   any libcw logs (because '--libcwflags' option is not provided).

5. Kill cwdaemon started in step 2. Start cwdaemon with '--libcwflags'
   option. Notice the 'd' (DEBUG) threshold for cwdaemon.

   ./src/cwdaemon --libcwflags=4294967295 -d null  -n -x p -y d

6. Send characters to cwdaemon using nc.

7. Confirm that cwdaemon's log output shows cwdaemon logs and show libcw logs
   with severity DEBUG or higher.

8. Kill cwdaemon started in step 4. Start cwdaemon with '--libcwflags'
   option. Notice the 'w' (WARNING) threshold for cwdaemon.

   ./src/cwdaemon --libcwflags=4294967295 -d null  -n -x p -y w

9. Send characters to cwdaemon using nc. Also send VOLUME Escape request
   (<ESC>g) with value out of range to trigger logging errors by cwdaemon.

10. Confirm that cwdaemon's log output shows cwdaemon logs. If there are any
    libcw logs, the logs have severity only WARNING or ERROR.

*Expected result*

1. libcw logs are visible in cwdaemon's log output only if `--libcwflags`
   flag is used

2. Threshold for libcw logs depends on value of '-y' flag passed to cwdaemon.

*Actual result*
PASS

***** PASS -f, --debugfile <output>

*Preconditions*

1. Connect USB-to-UART converter.


*Steps*

1. Confirm that cwdaemon logs messages to text file when started like this in
   non-forking mode:

   ./src/cwdaemon -n -x s -d /dev/ttyUSB0 -y d [-f/--debugfile] /tmp/non_forking.txt

2. Confirm that cwdaemon logs messages to text file when started like this in
   forking mode:

   ./src/cwdaemon -x s -d /dev/ttyUSB0 -y d [-f/--debugfile] /tmp/forking.txt

3. Confirm that cwdaemon logs messages to stdout when started like this:

   ./src/cwdaemon -n -x s -d /dev/ttyUSB0 -y d [-f/--debugfile] stdout 2>/tmp/stderr.txt

4. Confirm that cwdaemon logs messages to stderr when started like this:

   ./src/cwdaemon -n -x s -d /dev/ttyUSB0 -y d [-f/--debugfile] stderr 1>/tmp/stdout.txt

5. Confirm that cwdaemon rejects "syslog" as output of logs when started like
   this:

   ./src/cwdaemon -n -x s -d /dev/ttyUSB0 -y d [-f/--debugfile] syslog


*Expected result*
1. All of the behaviours from "Steps" are observed in running cwdaemon.
2. All of the behaviours from "Steps" are the same for short and long form of
   the command line option.


*Actual result*
PASS

**** PASS Misc tests

***** PASS reset register callback

*Preconditions*

1. Connect USB-to-UART converter.
2. Compile cwdaemon with functional tests enabled

   ./configure --enable-functional-tests
   make
   make check


*Test program*
./tests/functional_tests/unattended/reset_register_callback/test_program


*Steps*
1. Manually start the test
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*
PASS

***** PASS State of tty DTR and RTS pins during inactivity

*Purpose*

Confirm that cwdaemon doesn't change initial state of tty cwdevice's DTR and
RTS pins right after start.

Incorrect initialization of tty device has led to
https://github.com/acerion/cwdaemon/issues/12.

You can use cwdaemon 0.10.2 as a baseline. In a version following 0.10.2 a
change in ttys_init() was made that introduced an undesired behaviour of the
pins.


*Prerequisites*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. A device capable of showing a logical state on converter's pins: a
   multimeter or logical state analyzer.


*Steps*

1. Connect USB-to-UART converter to USB port.

2. Measure initial state of DTR and RTS pins

   Expectation: the state should be "high".

3. Start cwdaemon, specify a proper cwdevice through "-d" option

   ./src/cwdaemon -n -x s -ii -d ttyUSB0

4. Measure again the state of DTR and RTS pins.

   Expectation: cwdaemon should not change the state of the pins during start
   of cwdaemon and initialization of the cwdevice. The state on both pins
   should be "high".

5. Send some text to be played by cwdaemon

   nc -u 127.0.0.1 6789
   <Enter some text>

   After text is played and keyed on cwdevice, measure state of DTR and RTS
   pins.

   Expectation: after the text is played and keyed on cwdevice, the state of
   both pins should be "high".

6. Send "reset" command to cwdaemon

   Send "<ESC>0" Escape request to cwdaemon.

   nc -u 127.0.0.1 6789
   <Press ESC key>
   <Press "0" key>
   <Press Enter key>


   In cwdaemon's logs look for confirmation that reset is being performed:

   [INFO ] cwdaemon: requested resetting of parameters
   [INFO ] cwdaemon: resetting completed


   Wait for 2 seconds for the end of reset, then measure state of DTR and RTS
   pins.

   Expectation: after a reset is completed, the state of both pins should be
   "high".

7. Terminate cwdaemon

   Either press Ctrl-C while cwdaemon is running in foreground, or send
   "<ESC>5" Escape request to cwdaemon.

   Then measure state of DTR and RTS pins.

   Expectation: after an exit is performed, the state of both pins should be
   "high".


*Expected result*

All expectations from above steps are met.

*Actual result*
PASS

***** PASS tests/cwtest.pl

*Preconditions*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. cwdaemon must be started (automatically or manually), running on default
   port, with <platform's preferred sound system>.

   ./src/cwdaemon -n -x s -d ttyUSB0


*Test program*
cwtest.pl


*Steps*
1. Enter "tests" directory
2. Start the test script

   perl ./cwtest.pl --cycles 2

3. Observe test's output and cwdaemon's output.


*Expected result*
1. There are no errors in messages printed by cwdaemon.
2. There are no errors in messages printed by the test.


*Actual result*
PASS

***** PASS tests/cwtest_short_space.pl

*Preconditions*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. cwdaemon must be started (automatically or manually), running on default
   port, with <platform's preferred sound system>.

   ./src/cwdaemon -n -x s -d cuaU0


*Test program*
cwtest_short_space.pl


*Steps*
1. Enter "tests" directory
2. Start the test script

   perl ./cwtest_short_space.pl --cycles 20

3. Observe test's output and cwdaemon's output.


*Expected result*
1. There are no errors in messages printed by cwdaemon.
2. There are no errors in messages printed by the test.


*Actual result*
PASS

*** PASS gcov/lcov

*Purpose*

Confirm that it's possible to generate a coverage report for unit tests. It
is not expected that specific level of coverage is achieved.

Make sure that you can see coverage reports for source code of cwdaemon, i.e.
for files from ROOT/src.



*Prerequisites*

1. "lcov" package is installed.
2. The compilation and the coverage must be done with the same toolchain, and
   therefore on FreeBSD you may need to do this:

   export CC=clang

3. Connect an USB-to-UART cwdevice


*Steps*

The steps are copied from relevant sections in top-level README file:

1. ./configure --enable-functional-tests --enable-gcov
2. make gcov
3. <web browser> tests/coverage/unit_tests/daemon_<functionality>/index.html
4. make clean


*Expected results*

1. Summary of configuration step correctly shows enabled gcov,
2. There are no errors during generation of gcov/lcov report,
3. The reports are generated for source code from ROOT/src directory.
4. Contents of the report looks correct (no obvious problems with the
   report).
5. There are no errors during cleanup step.


*Actual result*
PASS

*** PASS memory tests with valgrind
**** PASS memory tests with valgrind for functional/unattended tests

*Goal*

Run all of cwdaemon's unattended functional tests in a setup where cwdaemon is
running under control of valgrind.


*Preconditions*

1. Connect USB-to-UART converter (a cwdevice).


*Steps*

1. Enable compilation of unattended functional tests

   ./configure --enable-functional-tests --enable-long-functional-tests

2. Compile cwdaemon

   make && make check

   You can terminate the "make check" command as soon as build system starts
   executing unit tests.

3. Prepare environment variables that will be used by all test programs
   executed with "make check"

   export CWDAEMON_TEST_SOUND_SYSTEM=null
   export CWDAEMON_TEST_SUPERVISOR=valgrind

4. Find all unattended tests

   find tests/functional_tests/unattended/ -name test_program

5. Run each binary by hand, one at a time.

   The purpose of running each test at a time is to get informed about a
   possible failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

tests/functional_tests/unattended/option_cwdevice_tty_lines/test_program   - PASS
tests/functional_tests/unattended/request_esc_exit/test_program            - PASS
tests/functional_tests/unattended/reset_register_callback/test_program     - PASS
tests/functional_tests/unattended/option_port/test_program                 - PASS
tests/functional_tests/unattended/request_plain/test_program               - PASS
tests/functional_tests/unattended/request_caret/test_program               - PASS
tests/functional_tests/unattended/request_esc_cwdevice/test_program        - PASS
tests/functional_tests/unattended/request_esc_reply/test_program           - PASS

**** PASS memory tests with valgrind for functional/supervised tests

*Goal*

Run all of cwdaemon's supervised functional tests in a setup where cwdaemon
is running under control of valgrind.


*Preconditions*

1. Connect USB-to-UART converter (a cwdevice).


*Steps*

1. Enable compilation of supervised functional tests

   ./configure --enable-functional-tests --enable-long-functional-tests

2. Compile cwdaemon

   make && make check

   You can terminate the "make check" command as soon as build system starts
   executing unit tests.

3. Prepare environment variables that will be used by all test programs
   executed with "make check"

   export CWDAEMON_TEST_SOUND_SYSTEM=null
   export CWDAEMON_TEST_SUPERVISOR=valgrind

4. Find all supervised tests

   find tests/functional_tests/supervised/ -name test_program

5. Run each binary by hand, one at a time.

   The purpose of running each test at a time is to get informed about a
   failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

tests/functional_tests/supervised/feature_multiple_requests/test_program  - SKIPPED (not a mature test yet)
tests/functional_tests/supervised/request_esc_sound_system/test_program   - PASS
tests/functional_tests/supervised/request_esc_port/test_program           - PASS

request_esc_sound_system switches to PulseAudio, and that results in lots of
Valgrind logs for PulseAudio library. I ignored these because they are most
probably due to memory/resource handling inside of PulseAudio library, and
have nothing to do with cwdaemon's resource management.

**** PASS memory tests with valgrind for fuzzing tests

*Goal*

Run all of cwdaemon's fuzzing tests in a setup where cwdaemon is running
under control of valgrind.


*Steps*

1. Confirm that fuzzing tests done few sections above were already executed
   with cwdaemon being under control of valgrind.

*Expected result*

No additional test is needed here because the primary fuzzing tests already
use valgrind.

*Actual result*

PASS

**** PASS memory tests with valgrind for unit tests

*Goal*

Run all of cwdaemon's unit tests in a setup where unit tests are running
under control of valgrind.


*Steps*

1. Enable compilation of unit tests (unit tests of cwdaemon only)

   make clean
   ./configure

2. Compile cwdaemon and unit tests

   make && make check

3. Find all unit test binaries

   find tests/unit_tests/* -type f  -executable

4. Run each binary by hand, one at a time.

   /usr/bin/valgrind -s --leak-check=full --show-leak-kinds=all --track-fds=yes <path to test binary>

   The purpose of running each test at a time is to get informed about a
   failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

tests/unit_tests/daemon_options   - PASS
tests/unit_tests/daemon_sleep     - PASS
tests/unit_tests/daemon_utils     - PASS

** PASS Alpine Linux 3.20

OS: Alpine Linux 3.20
C compiler: gcc (Alpine 13.2.1_git20240309) 13.2.1 20240309
libc: musl-1.2.5-r0
unixcw/libcw: 3.6.1/8:0:0

*** PASS tests of build system
**** PASS make distcheck
*Steps*

Run the steps in main directory of project.

On FreeBSD you may need to use "gmake" (GNU make) instead of FreeBSD's
default "make".

1. ./configure
2. make distcheck

*Expected result*
1. Distribution package is built correctly (no errors during creation of
   archive).
2. Automatic tests executed during 'make distcheck' pass.
3. The distribution package contains all files intended for distribution (use
   your best judgement here).

*Actual result*

PASS

I had to do "autoreconf && automake" first, otherwise make distcheck would
report problems with some of source files.

**** PASS ./configure flags

Test that all flags supported by ./configure script produce "buildable" code.

*Steps*
1. Run "make clean && ./configure && make && make check",
2. Run "make clean && ./configure --enable-functional-tests && make && make check",
3. Run "make clean && ./configure --enable-functional-tests --enable-long-functional-tests && make && make check",

*Expected result*

Each of the above tests results in correct build: binaries are built
correctly, including binaries for all kinds of tests.

You don't have to do full tests with these test binaries, just check that
they are there.

*Actual result*

PASS

**** PASS regression with make distcheck

Confirm that failure of 'make distcheck' after specific sequence of steps does not occur.

On Alpine 3.17 and with certain version of tests/unit_tests/Makefile.am, the
steps ended with compilation error, and distribution package was not created.

The error message was:

gcc  -g -O2 -Wall -Wextra -pedantic -Wswitch-default -Wimplicit-function-declaration -Wswitch-enum -Wfloat-equal -Wpointer-arith -Wcast-qual -Wwrite-strings -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wredundant-decls -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict -Wnull-dereference -Wjump-misses-init -Wdouble-promotion -Wshadow -Wformat=2 -std=c99   -o utils ../../src/utils-utils.o utils-utils.o  
/usr/lib/gcc/x86_64-alpine-linux-musl/12.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find utils-utils.o: No such file or directory
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:328: utils] Error 1
make[4]: Leaving directory '/home/acerion/cwdaemon/git_repo/cwdaemon-0.12.0/_build/sub/tests/unit_tests'



*Steps*
make distclean
autoreconf
automake
./configure
make
make check
make distcheck  # <---- In this step a compilation error was occurring


*Expected result*
Distribution archive is created correctly.

*Actual result*

PASS

*** PASS unit tests

Execute tests implemented in tests/unit_tests.

*Steps*
1. ./configure --enable-functional-tests
2. make check
3. Wait for all unit tests to be executed. You can terminate the tests as
   soon as unit tests complete, and functional tests start being executed.

*Expected result*
All unit tests should have "PASS" status in output of "make check" command.

This shows examples of logs for unit tests (notice the "unit_tests"
directory):
PASS: unit_tests/daemon_utils
PASS: unit_tests/daemon_options
PASS: unit_tests/tests_random
PASS: unit_tests/tests_time_utils

*Notes*

Count of unit tests depends on whether you pass "--enable-functional-tests"
to ./configure or not. Test both cases.

*Actual result*

PASS

 - With functional tests not enabled:

PASS: unit_tests/daemon_utils
PASS: unit_tests/daemon_options
PASS: unit_tests/daemon_sleep

 - With functional tests enabled:

PASS: unit_tests/daemon_utils
PASS: unit_tests/daemon_options
PASS: unit_tests/daemon_sleep
PASS: unit_tests/tests_random
PASS: unit_tests/tests_string_utils
PASS: unit_tests/tests_time_utils
PASS: unit_tests/tests_morse_receiver
PASS: unit_tests/tests_events

*** PASS fuzzing tests
**** PASS simple fuzzing test

*Purpose*

Run simple fuzzing test to catch memory errors.

Run the test early in testing phase, certainly before doing manual tests. Let
an unattended fuzzing test detect problems early.

*Preconditions*

1. Be sure to use large count of iterations in the test: set value of
   "n_iters" in test_run() to high value.

2. Be sure to run the test under valgrind: set "supervisor_id" to
   "supervisor_id_valgrind" in options passed to server_start().

3. Be sure to have all tests enabled in "g_test_cases[]".

4. Plug-in an USB-to-UART converter.


*Steps*

1. Compile cwdaemon package with fuzzing test enabled:

   ./configure --enable-functional-tests
   make
   make check

   You can interrupt the automatic execution of unit and functional tests.

2. Run test program: tests/fuzzing/simple/test_program
3. Look at summary of valgrind test, looking for information about memory
   errors.


*Expected result*

1. valgrind reports no memory leaks (with possible exception of memory leak
   described in ticket R0018).

2. valgrind reports no memory errors of other types.

3. cwdaemon's test framework doesn't report any errors on its own

*Actual result*

PASS

*** PASS functional tests

PASS: most important tests are a PASS, rest of them are SKIPPED for reason
given in each skipped test.

**** PASS unattended short tests

*Goal*
Run unattended tests, but without "long" variants. Run relatively short tests
and look at final result to have some indication of general status of tests
before delving deeper into testing procedures that require more attention.


*Preconditions*
1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests
3. make


*Steps*
1. Run "make check"
2. Let the unattended tests run and complete


*Expectations*
The unattended tests executed through "make check" pass. We can now perform
tests that require more attention from tester.


*Actual result*

PASS. I needed to make sure that TESTS_WPM_MAX is being used consistently
throughout code of functional tests for setting constant value of wpm.

**** PASS unattended long tests

*Goal*
Run unattended tests, but this time also with "long" variants. Run relatively
long tests and look at final result to have some indication of general status
of tests before delving deeper into testing procedures that require more
attention.


*Preconditions*
1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests
3. make


*Steps*
1. Run "make check"
2. Let the unattended tests run and complete


*Expectations*
The unattended tests executed through "make check" pass. We can now perform
tests that require more attention from tester.


*Actual result*

PASS

I needed to apply this patch to make the tests pass on a specific weak
machine (a netbook with Atom CPU). The patch only changes some parameters of
tests, not of cwdaemon, so I think that it's fair to give here a PASS. It's
the tests' receiver failing, not the cwdaemon.

The changed values of intervals are magic values at this point, without good
rationale behind them, so the changes of the two values may be something that
I would do in the future anyway (maybe just to a smaller degree).


diff --git a/tests/functional_tests/supervised/request_esc_port/shared.c b/tests/functional_tests/supervised/request_esc_port/shared.c
index 49fada4..a690264 100644
--- a/tests/functional_tests/supervised/request_esc_port/shared.c
+++ b/tests/functional_tests/supervised/request_esc_port/shared.c
@@ -167,7 +167,7 @@ static int test_setup(server_t * server, client_t * client, morse_receiver_t * m
 {
 	// We want test cases to be executed rather quickly, so play the Morse
 	// code rather quickly.
-	const int wpm = 20;
+	const int wpm = TESTS_WPM_MAX;
 
 	// Prepare local test instance of cwdaemon server.
 	const server_options_t server_opts = {
diff --git a/tests/functional_tests/unattended/request_caret/shared.c b/tests/functional_tests/unattended/request_caret/shared.c
index b798412..7310ec7 100644
--- a/tests/functional_tests/unattended/request_caret/shared.c
+++ b/tests/functional_tests/unattended/request_caret/shared.c
@@ -158,7 +158,7 @@ static int test_setup(server_t * server, client_t * client, morse_receiver_t * m
 	   quickly to make the test short. Using "25" because at "30" there are
 	   too many errors. TODO (acerion) 2024.05.01: fix receiving and increase
 	   the speed. */
-	const int wpm = 25;
+	const int wpm = TESTS_WPM_MAX;
 
 	/* Prepare local test instance of cwdaemon server. */
 	const server_options_t server_opts = {
diff --git a/tests/functional_tests/unattended/request_esc_reply/shared.c b/tests/functional_tests/unattended/request_esc_reply/shared.c
index ad0d132..f3137be 100644
--- a/tests/library/client.c
+++ b/tests/library/client.c
@@ -64,7 +64,7 @@
 
 
 // Poll interval for client's receiver polling [milliseconds].
-#define RECEIVE_THREAD_INTERVAL_MS     1000
+#define RECEIVE_THREAD_INTERVAL_MS     20
 
 // How long to wait for termination of receive thread [milliseconds].
 #define RECEIVE_THREAD_STOP_WAIT_MS   (RECEIVE_THREAD_INTERVAL_MS + (RECEIVE_THREAD_INTERVAL_MS * 0.2))
diff --git a/tests/library/cwdevice_observer.c b/tests/library/cwdevice_observer.c
index 411128a..8992629 100644
--- a/tests/library/cwdevice_observer.c
+++ b/tests/library/cwdevice_observer.c
@@ -57,7 +57,7 @@
 /// TODO (acerion) 2024.07.26: value of 100us results in rather high CPU
 /// usage (8-10%) on a weak machine. See if that short interval and that high
 /// CPU usage is justified.
-#define KEY_SOURCE_DEFAULT_INTERVAL_US 100
+#define KEY_SOURCE_DEFAULT_INTERVAL_US 20
 
 
 
diff --git a/tests/library/test_defines.h b/tests/library/test_defines.h
index e57e2ae..9b70b89 100644
--- a/tests/library/test_defines.h
+++ b/tests/library/test_defines.h
@@ -104,7 +104,7 @@
 
 /* Lower and upper limit of Morse code speeds used in functional tests. */
 #define TESTS_WPM_MIN 10
-#define TESTS_WPM_MAX 20 /* At 25 we start to have receive errors. TODO (acerion) 2024.04.19: fix the receiving and increase the limit. */
+#define TESTS_WPM_MAX 12 /* At 25 we start to have receive errors. TODO (acerion) 2024.04.19: fix the receiving and increase the limit. */
 #define TESTS_WPM_DEFAULT 10 /* TODO (acerion) 2024.04.19: increase the default at some point in the future. */
 
 



**** SKIPPED PLAIN request

*Preconditions*
1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests
3. make && make check
4. Interrupt the build process when tests start being executed


*Test program*
./tests/functional_tests/unattended/request_plain/test_program


*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test


*Expectations*
Final result of the test is a "PASS"


*Actual result*

SKIPPED: such tests were already done on FreeBSD and on Debian Linux. I won't
be repeating them on Alpine.

**** SKIPPED CARET request

*Preconditions*
1. Plug in USB-to-UART converter to USB socket.
2. ./configure --enable-functional-tests --enable-long-functional-tests
3. make && make check
4. Interrupt the build process when tests start being executed


*Test program*
tests/functional_tests/unattended/request_caret/test_program


*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test


*Expectations*
Final result of the test is a "PASS"


*Actual result*

SKIPPED: such tests were already done on FreeBSD and on Debian Linux. I won't
be repeating them on Alpine.

**** SKIPPED Escape requests

SKIPPED: such tests were already done on FreeBSD and on Debian Linux. I won't
be repeating them on Alpine.

***** TODO RESET        '0'

*Preconditions*

1. Connect USB-to-UART converter.


*Info*

The following parameters are reset with RESET Escape request:
 - Morse speed,
 - tone (frequency) of played Morse code,
 - volume of played Morse code,
 - sound system used to play Morse code,
 - PTT delay
 - weighting of Morse code,
 - state of libcw,
 - word mode (to false),
 - state of pins of cwdevice,
 - cwdaemon's internal PTT flags


*Steps*
1. in terminal A start cwdaemon with specific values of some of parameters.
   Those will be the initial/default parameters.

   ./src/cwdaemon -n -x s --wpm 15 --tone 1000 --volume 100 --weighting 50

2. In terminal B start "nc" as a client of cwdaemon:

   nc -u 127.0.0.1 6789

3. In terminal B send some text to confirm that cwdaemon can play it with
   initial/default parameters.

4. In terminal B send few Escape requests to modify parameters of cwdaemon:
   - Morse speed: change to 8 (Esc 2)
   - tone: change to 400 (Esc 3)
   - volume: change to 20 (Esc g)
   - weighting: change to 0 (Esc 7)

5. In terminal B send some text to confirm that cwdaemon can play it with
   modified parameters.

6. In terminal B send RESET Escape request to change cwdaemon's parameters
   back to their initial/default values.

7. In terminal B send some text to confirm that cwdaemon is playing the text
   with initial/default parameters.


*Expected result*
1. RESET Escape sent in step 6 does reset parameters to their initial values,
   which is confirmed by how cwdaemon plays a request in step 7.

*Actual result*

***** TODO SPEED        '2'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc2.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_esc2.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors

*Actual result*

***** TODO TONE         '3'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc3.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_esc3.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors

*Actual result*

***** TODO ABORT        '4'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc64.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_esc64.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors
3. Text played by cwdaemon can be interrupted by ABORT Escape request when
   cwdaemon works in interruptible mode.
4. Text played by cwdaemon cannot be interrupted by ABORT Escape request when
   cwdaemon works in non-interruptible mode.
5. cwdaemon behaves as described in points 3 and 4 even when invalid ABORT
   Escape request is being sent, i.e. when the value of the request is
   non-empty.

*Actual result*

***** TODO EXIT         '5'

*Preconditions*
1. Connect USB-to-UART converter.

*Test program*
./tests/functional_tests/unattended/request_esc_exit/test_program

*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*

***** TODO WORD_MODE    '6'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc64.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_esc64.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors
3. cwdaemon is being switched between word-mode (non-interruptible mode) and
   regular mode (interruptible mode).

   A current mode can be recognized by cwdaemon's response to ABORT Escape
   request.

   Played text cannot (in word-mode) or can (in regular mode) be interrupted
   by ABORT Escape request.

*Actual result*

***** TODO WEIGHTING    '7'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esc7.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_esc7.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. Weighting of dits and dashes changes as expected,
3. cwdaemon doesn't report any unexpected errors.

*Actual result*

***** TODO CWDEVICE     '8'

*Preconditions*
1. Attach two USB-to-UART devices acting as two cwdevices: default one, and
   non-default one.

*Test program*
./tests/functional_tests/unattended/request_esc_cwdevice/test_program

*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*

***** TODO PORT         '9'

*Preconditions*
1. Connect USB-to-UART converter.

*Test program*
./tests/functional_tests/supervised/request_esc_port/test_program

*Steps*
1. Manually start the test program
2. Read the banner text displayed at the beginning of test,
3. Observe the logs displayed during execution of test,
4. Wait for final status of the test.

*Expectations*
1. Final result of the test is a "PASS".
2. None of characteristics of played sounds are changing between iterations
   of test.
3. You can see the following message in logs of cwdaemon when PORT Escape
   request is sent to cwdaemon:

   [WW] cwdaemon: obsolete request "9" (change network port), ignoring


*Actual result*

***** TODO PTT_STATE    'a'

*Preconditions*

1. Prepare a multimeter or logic state analyzer.
2. Read information at the beginning of tests/cwtest_esca.pl script about 3
   modes.
3. Connect USB-to-UART converter.
4. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s --cwdevice <cwdevice>
5. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_esca.pl

*Steps*
1. Connect the multimeter to PTT pin of the USB-to-UART converter.
2. In terminal B run the test program

   perl ./cwtest_esca.pl --mode X

   Do this with value of mode (X) equal to 0, 1, and 2.
3. Observe output of test in terminal B
4. Observe readings on the multimeter, showing state of PTT pin (ON or OFF).


*Expected result*
1. State of PTT pin changes at all, as shown by multimeter's reading,
2. State of PTT pin changes according to mode used by the test script.

*Actual result*

***** TODO SSB_WAY      'b'

*Expected result*
Test passes


*Actual result*

***** TODO TUNE         'c'
****** TODO Perl script

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escc.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_escc.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B
4. Count the duration of each generated tuning tone.


*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors,
3. Each generated tuning tone lasted for expected count of seconds.


*Actual result*

****** TODO Manual test, with observing PTT Flag

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s -iii -t 1
3. In another terminal (terminal B) run nc:
   nc -u 127.0.0.1 6789


*Steps*
1. In terminal B request long tuning time
   ^[c9
2. In terminal A observe that PTT pin is raised:

   [II] cwdaemon: -------------------
   [II] cwdaemon: received Escape request: "<ESC>c" / "<ESC>0x63"
   [II] cwdaemon: requested tuning time [s]: "9"
   [II] cwdaemon: PTT (TUNE) on
   [DD] cwdaemon: PTT flag +PTT_ACTIVE_AUTO (0x01/Ame)
   [DD] cwdaemon: keying event 1

3. After the tuning time elapses, observe in terminal A that PTT pin is
   lowered:

   [II] cwdaemon: low TQ callback: start, TQ len = 1, PTT flag = 0x01/Ame
   [II] cwdaemon: low TQ callback: branch 1, PTT flag = 0x01/Ame
   [DD] cwdaemon: PTT flag = 0 (0x00/ame)
   [II] cwdaemon: PTT (auto) off
   [II] cwdaemon: low TQ callback: end, TQ len = 1, PTT flag = 0x00/ame
   [DD] cwdaemon: keying event 0


*Expected result*
1. PTT pin is raised and lowered as expected,
2. cwdaemon doesn't report any unexpected errors


*Actual result*

***** TODO TX_DELAY     'd'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escd.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_escd.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. cwdaemon doesn't report any unexpected errors.

*Actual result*

***** TODO BAND_SWITCH  'e'

*Expected result*
Test passes


*Actual result*

***** TODO SOUND_SYSTEM 'f'

The test may be trying to switch to invalid sound system "?"/"<NONE>". Such
attempts will be ignored by cwdaemon, and cwdaemon SHOULD continue to use
previous sound system.

*Preconditions*
1. Attach USB-to-UART device acting as cwdevice.

*Test program*
./tests/functional_tests/supervised/request_esc_sound_system/test_program

*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*

***** TODO VOLUME       'g'

*Preconditions*

1. Connect USB-to-UART converter.
2. In one terminal (terminal A) start a test instance of cwdaemon:
   ./src/cwdaemon -n -x s
3. In another terminal (terminal B) enter into "tests" dir.

*Test program*
cwtest_escg.pl

*Steps*
1. In terminal B run the test program
   perl ./cwtest_escg.pl
2. Observe output of cwdaemon in terminal A
3. Observe output of test in terminal B

*Expected result*
1. Test passes without errors,
2. Volume of sounds produced by cwdaemon changes as expected,
3. cwdaemon doesn't report any unexpected errors.

*Actual result*

***** TODO REPLY        'h'

*Test program*
./tests/functional_tests/unattended/request_esc_reply/test_program

*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*

**** SKIPPED Command line options

SKIPPED: such tests were already done on FreeBSD and on Debian Linux. I won't
be repeating them on Alpine.

***** TODO -h, --help

***** TODO -V, --version

***** TODO -d, --cwdevice <device>

*Preconditions*
1. Connect two cwdevices


*Steps*
1. In one terminal (terminal A) start a test instance of cwdaemon:

   ./src/cwdaemon -n -x s --cwdevice <cwdevice-1>

2. In another terminal (terminal B) send some text using nc. As the text is
   being played, confirm that cwdaemon is toggling "keying" pin on
   <cwdevice-1>, but not in <cwdevice-2>.

3. In terminal A stop cwdaemon and start it again, but this time with the
   second cwdevice.

   ./src/cwdaemon -n -x s --cwdevice <cwdevice-2>

4. In another terminal (terminal B) send some text using nc. As the text is
   being played, confirm that cwdaemon is toggling "keying" pin on
   <cwdevice-2>, but not in <cwdevice-1>.


*Expected result*
1. cwdaemon is using the right cwdevice, as specified in command line.


*Actual result*

***** TODO -o, --options <option>

*Test program*
./tests/functional_tests/unattended/option_cwdevice_tty_lines/test_program

*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test
4. Manually run at least one test with the long "--options" form of the
   command line option.
5. Try passing invalid values of "key=" or "ptt=" keywords in command line.


*Expectations*
1. Final result of the test is a "PASS".
2. Handling of long "--options" option work.
3. Invalid values of "key=" or "ptt=" keywords are rejected.


*Actual result*

***** TODO -n, --nofork

*Preconditions*

Find out the syslog output to which daemon which output a "daemon" processes
are logging.

On FreeBSD, per /etc/syslog.conf, the INFO messages go to
/var/log/daemon.log.

On Linux, you will be able to see messages in output of "journalctl -f":
<netkeyer[5093]: received request: "is this syslog?">

****** TODO do not fork

*Steps*

1. Run cwdaemon in non-forking mode

   ./src/cwdaemon [-n/--nofork] -x p -ii

2. In another terminal send some PLAIN message to cwdaemon.


*Expected result*

1. Observe that cwdaemon is still attached to current terminal (it didn't go
   to background).
2. Observe that cwdaemon is logging messages to stdout


*Actual result*

****** TODO do fork

*Steps*

1. Confirm that no other cwdaemon is already running

   ps -A | grep cwdaemon

2. Run cwdaemon in forking mode

   ./src/cwdaemon -x p -ii

3. Confirm that cwdaemon process is present with this command:

   ps -A | grep cwdaemon

4. In another terminal send some PLAIN message to cwdaemon.

5. In another terminal send EXIT Escape request (<ESC>5).

6. Confirm that cwdaemon process is no longer present (because it exited)
   with this command:

   ps -A | grep cwdaemon


*Expected result*

1. After step 2 the cwdaemon is no longer attached to current terminal (it
   went to background).
2. In step 4 cwdaemon is not logging messages to stdout.
3. In step 4 cwdaemon is logging messages to syslog instead.
4. In step 4 cwdaemon is playing Morse code using sound system.


*Actual result*

***** TODO -p, --port <port>

*Test program*
./tests/functional_tests/unattended/option_port/test_program


*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test


*Expectations*
Final result of the test is a "PASS"


*Actual result*

***** TODO -s, --wpm <speed>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.


*Actual result*

***** TODO -t, --pttdelay <time>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon.
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*

***** TODO -x, --system <sound system>

*Preconditions*
1. Connect a cwdevice


*Info*
cwdaemon may fail to run with OSS on Linux, and detecting the failure by
cwdaemon may last ~15 seconds.


*Steps*
1. Run cwdaemon with different sound systems: one of 'n', 'c', 'o', 'a', 'p',
   's'.
2. Run cwdaemon with invalid sound system, e.g. 'u'.
3. Observe if cwdaemon exits due to unsupported sound system, or if it starts
   correctly.
4. If it starts correctly, try to sending some text through "nc".


*Expected results*
1. cwdaemon doesn't start correctly if a sound system not supported on the
   test platform is specified.
2. cwdaemon starts correctly if a sound system supported on the test platform
   is specified.
3. If cwdaemon starts correctly, it plays a text using specified sound
   system.
4. If cwdaemon starts correctly, Morse code is keyed on cwdevice's 'key' pin.
5. Both short and long form of the command line option works the same.


*Actual result*

***** TODO -v, --volume <volume>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*

***** TODO -w, --weighting <weight>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.
4. cwdaemon accepts negative values (you may need to put them in quotes)

*Actual result*

***** TODO -T, --tone <tone>

*Expected result*
1. It's possible to set parameter using short and long form of the option.
2. Values in range are accepted by cwdaemon (confirmed by playing some text).
3. Values out of range are rejected - cwdaemon prints error message and exits.

*Actual result*

***** TODO -i

*Preconditions*

1. Connect USB-to-UART converter.


*Steps*

1. In one terminal (terminal A) start cwdaemon without "-i" flags:

   ./src/cwdaemon -n -x s -d <cwdevice>

2. In another terminal (terminal B) send a valid TONE Escape request
   (<ESC>3), send an invalid TONE Escape request, and then play some text.

3. In terminal A observe that with no "-i" flags specified in command line,
   cwdaemon prints only errors and warnings

4. In terminal A do a series of starts of cwdaemon with increasing amount of
   "-i" flags. In terminal B send a TONE Escape request, then play some text,
   and then in terminal A observe increasing verbosity of cwdaemon's logs.

   "-i"     ->  notice
   "-ii"    ->  info
   "-iii"   ->  debug
   "-iiii"  ->  still debug


*Expected result*
1. Verbosity of logs in cwdaemon increases with count of "i" options passed
   in command line.


*Actual result*

***** TODO -y, --verbosity <threshold>

*Preconditions*

1. Connect USB-to-UART converter.

*Steps*

1. In one terminal (terminal A) start cwdaemon without "-y"/"--verbosity"
   flag:

   ./src/cwdaemon -n -x s -d <cwdevice>

2. In another terminal (terminal B) send a valid TONE Escape request
   (<ESC>3), send an invalid TONE Escape request, and then play some text.

3. In terminal A observe that with no "-y"/"--verbosity" flag specified in
   command line, cwdaemon prints only errors and warnings.

4. In terminal A do a series of starts of cwdaemon with increasing a
   verbosity level passed to "-y"/"--verbosity" flag (n (none), e (error), w
   (warning), i (information), d (debug)). In terminal B send a TONE Escape
   request, then play some text, and then in terminal A observe increasing
   verbosity of cwdaemon's logs.

   In particular confirm that with "n" flag cwdaemon doesn't even print
   errors.


*Expected result*

1. Verbosity of logs in cwdaemon increases with level of verbosity passed to
   "-y"/"--verbosity" in command line.


*Actual result*

***** TODO -I, --libcwflags <flags>

*Goal*
Confirm that option that specifies debug flags for libcw is working.


*Info*
In this test cwdaemon is executed:
 - in non-daemonized mode to observe console logs;
 - with null keying device because cwdevice is not essential for this test.


*Steps*

1. In terminal A, open connection to cwdaemon.

   nc -u localhost 6789
   OR
   nc -u 127.0.0.1 6789

2. In terminal B, run cwdaemon without the tested flag. Notice the 'd'
   (DEBUG) threshold for cwdaemon.

   ./src/cwdaemon -d null -n -x s -y d

3. In terminal A, send some characters to cwdaemon with nc.

4. Confirm that cwdaemon's log output shows cwdaemon logs, but doesn't show
   any libcw logs (because '--libcwflags' option is not provided).

5. Kill cwdaemon started in step 2. Start cwdaemon with '--libcwflags'
   option. Notice the 'd' (DEBUG) threshold for cwdaemon.

   ./src/cwdaemon --libcwflags=4294967295 -d null  -n -x p -y d

6. Send characters to cwdaemon using nc.

7. Confirm that cwdaemon's log output shows cwdaemon logs and show libcw logs
   with severity DEBUG or higher.

8. Kill cwdaemon started in step 4. Start cwdaemon with '--libcwflags'
   option. Notice the 'w' (WARNING) threshold for cwdaemon.

   ./src/cwdaemon --libcwflags=4294967295 -d null  -n -x p -y w

9. Send characters to cwdaemon using nc. Also send VOLUME Escape request
   (<ESC>g) with value out of range to trigger logging errors by cwdaemon.

10. Confirm that cwdaemon's log output shows cwdaemon logs. If there are any
    libcw logs, the logs have severity only WARNING or ERROR.


*Expected result*

1. libcw logs are visible in cwdaemon's log output only if `--libcwflags`
   flag is used

2. Threshold for libcw logs depends on value of '-y' flag passed to cwdaemon.

*Actual result*

***** TODO -f, --debugfile <output>

*Preconditions*

1. Connect USB-to-UART converter.


*Steps*

1. Confirm that cwdaemon logs messages to text file when started like this in
   non-forking mode:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] /tmp/non_forking.txt

2. Confirm that cwdaemon logs messages to text file when started like this in
   forking mode:

   ./src/cwdaemon -x s -d /dev/cuaU0 -y d [-f/--debugfile] /tmp/forking.txt

3. Confirm that cwdaemon logs messages to stdout when started like this:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] stdout 2>/tmp/stderr.txt

4. Confirm that cwdaemon logs messages to stderr when started like this:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] stderr 1>/tmp/stdout.txt

5. Confirm that cwdaemon rejects "syslog" as output of logs when started like
   this:

   ./src/cwdaemon -n -x s -d /dev/cuaU0 -y d [-f/--debugfile] syslog


*Expected result*
1. All of the behaviours from "Steps" are observed in running cwdaemon.
2. All of the behaviours from "Steps" are the same for short and long form of
   the command line option.


*Actual result*

**** SKIPPED Misc tests

SKIPPED: such tests were already done on FreeBSD and on Debian Linux. I won't
be repeating them on Alpine.

***** TODO reset register callback

*Preconditions*

1. Connect USB-to-UART converter.
2. Compile cwdaemon with functional tests enabled

   ./configure --enable-functional-tests
   make
   make check


*Test program*
./tests/functional_tests/unattended/reset_register_callback/test_program


*Steps*
1. Manually start the test program
2. Observe the logs
3. Wait for final status of the test

*Expectations*
Final result of the test is a "PASS"

*Actual result*

***** TODO State of tty DTR and RTS pins during inactivity

*Purpose*

Confirm that cwdaemon doesn't change initial state of tty cwdevice's DTR and
RTS pins right after start.

Incorrect initialization of tty device has led to
https://github.com/acerion/cwdaemon/issues/12.

You can use cwdaemon 0.10.2 as a baseline. In a version following 0.10.2 a
change in ttys_init() was made that introduced an undesired behaviour of the
pins.


*Prerequisites*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. A device capable of showing a logical state on converter's pins: a
   multimeter or logical state analyzer.


*Steps*

1. Connect USB-to-UART converter to USB port.

2. Measure initial state of DTR and RTS pins

   Expectation: the state should be "high".

3. Start cwdaemon, specify a proper cwdevice through "-d" option

   ./src/cwdaemon -n -x s -ii -d ttyUSB0

4. Measure again the state of DTR and RTS pins.

   Expectation: cwdaemon should not change the state of the pins during start
   of cwdaemon and initialization of the cwdevice. The state on both pins
   should be "high".

5. Send some text to be played by cwdaemon

   nc -u 127.0.0.1 6789
   <Enter some text>

   After text is played and keyed on cwdevice, measure state of DTR and RTS
   pins.

   Expectation: after the text is played and keyed on cwdevice, the state of
   both pins should be "high".

6. Send "reset" command to cwdaemon

   Send "<ESC>0" Escape request to cwdaemon.

   nc -u 127.0.0.1 6789
   <Press ESC key>
   <Press "0" key>
   <Press Enter key>


   In cwdaemon's logs look for confirmation that reset is being performed:

   [INFO ] cwdaemon: requested resetting of parameters
   [INFO ] cwdaemon: resetting completed


   Wait for 2 seconds for the end of reset, then measure state of DTR and RTS
   pins.

   Expectation: after a reset is completed, the state of both pins should be
   "high".

7. Terminate cwdaemon

   Either press Ctrl-C while cwdaemon is running in foreground, or send
   "<ESC>5" Escape request to cwdaemon.

   Then measure state of DTR and RTS pins.

   Expectation: after an exit is performed, the state of both pins should be
   "high".


*Expected result*

All expectations from above steps are met.

*Actual result*

***** TODO tests/cwtest.pl

*Preconditions*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. cwdaemon must be started (automatically or manually), running on default
   port, with <platform's preferred sound system>.

   ./src/cwdaemon -n -x s -d cuaU0


*Test program*
cwtest.pl


*Steps*
1. Enter "tests" directory
2. Start the test script

   perl ./cwtest.pl --cycles 2

3. Observe test's output and cwdaemon's output.


*Expected result*
1. There are no errors in messages printed by cwdaemon.
2. There are no errors in messages printed by the test.


*Actual result*

***** TODO tests/cwtest_short_space.pl

*Preconditions*

1. USB-to-UART converter that exposes DTR pin and RTS pin.
2. cwdaemon must be started (automatically or manually), running on default
   port, with <platform's preferred sound system>.

   ./src/cwdaemon -n -x s -d cuaU0


*Test program*
cwtest_short_space.pl


*Steps*
1. Enter "tests" directory
2. Start the test script

   perl ./cwtest_short_space.pl --cycles 20

3. Observe test's output and cwdaemon's output.


*Expected result*
1. There are no errors in messages printed by cwdaemon.
2. There are no errors in messages printed by the test.


*Actual result*

*** SKIPPED gcov/lcov

*Purpose*

Confirm that it's possible to generate a coverage report for unit tests. It
is not expected that specific level of coverage is achieved.

Make sure that you can see coverage reports for source code of cwdaemon, i.e.
for files from ROOT/src.


*Prerequisites*

1. "lcov" package is installed.
2. The compilation and the coverage must be done with the same toolchain, and
   therefore on FreeBSD you may need to do this:

   export CC=clang

3. Connect an USB-to-UART cwdevice


*Steps*

The steps are copied from relevant sections in top-level README file:

1. ./configure --enable-functional-tests --enable-gcov
2. make gcov
3. <web browser> tests/coverage/unit_tests/daemon_<functionality>/index.html
4. make clean


*Expected results*

1. Summary of configuration step correctly shows enabled gcov,
2. There are no errors during generation of gcov/lcov report,
3. Contents of the report looks correct (no obvious problems with the
   report).
4. There are no errors during cleanup step.


*Actual result*

SKIPPED: such tests were already done on FreeBSD and on Debian Linux. I won't
be repeating them on Alpine.

*** SKIPPED memory tests with valgrind

SKIPPED: such tests were already done on FreeBSD and on Debian Linux. I won't
be repeating them on Alpine (even though Alpine uses its own libc).

**** TODO memory tests with valgrind for functional/unattended tests

*Goal*

Run all of cwdaemon's unattended functional tests in a setup where cwdaemon is
running under control of valgrind.


*Preconditions*

1. Connect USB-to-UART converter (a cwdevice).


*Steps*

1. Enable compilation of unattended functional tests

   ./configure --enable-functional-tests --enable-long-functional-tests

2. Compile cwdaemon

   make && make check

   You can terminate the "make check" command as soon as build system starts
   executing unit tests.

3. Prepare environment variables that will be used by all test programs
   executed with "make check"

   export CWDAEMON_TEST_SOUND_SYSTEM=null
   export CWDAEMON_TEST_SUPERVISOR=valgrind

4. Find all unattended tests

   find tests/functional_tests/unattended/ -name test_program

5. Run each binary by hand, one at a time.

   The purpose of running each test at a time is to get informed about a
   possible failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

**** TODO memory tests with valgrind for functional/supervised tests

*Goal*

Run all of cwdaemon's supervised functional tests in a setup where cwdaemon
is running under control of valgrind.


*Preconditions*

1. Connect USB-to-UART converter (a cwdevice).


*Steps*

1. Enable compilation of supervised functional tests

   ./configure --enable-functional-tests --enable-long-functional-tests

2. Compile cwdaemon

   make && make check

   You can terminate the "make check" command as soon as build system starts
   executing unit tests.

3. Prepare environment variables that will be used by all test programs
   executed with "make check"

   export CWDAEMON_TEST_SOUND_SYSTEM=null
   export CWDAEMON_TEST_SUPERVISOR=valgrind

4. Find all supervised tests

   find tests/functional_tests/supervised/ -name test_program

5. Run each binary by hand, one at a time.

   The purpose of running each test at a time is to get informed about a
   failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

**** TODO memory tests with valgrind for fuzzing tests

*Goal*

Run all of cwdaemon's fuzzing tests in a setup where cwdaemon is running
under control of valgrind.


*Steps*

1. Confirm that fuzzing tests done few sections above were already executed
   with cwdaemon being under control of valgrind.

*Expected result*

No additional test is needed here because the primary fuzzing tests already
use valgrind.

*Actual result*

**** TODO memory tests with valgrind for unit tests

*Goal*

Run all of cwdaemon's unit tests in a setup where unit tests are running
under control of valgrind.


*Steps*

1. Enable compilation of unit tests (unit tests of cwdaemon only)

   make clean
   ./configure

2. Compile cwdaemon and unit tests

   make && make check

3. Find all unit test binaries

   find tests/unit_tests/* -type f  -executable

4. Run each binary by hand, one at a time.

   /usr/local/bin/valgrind -s --leak-check=full --show-leak-kinds=all --track-fds=yes <path to test binary>

   The purpose of running each test at a time is to get informed about a
   failure early.

   1. After each test is completed, confirm that no errors are present in
      summary of the test, in one of these sections:

      1. "FILE DESCRIPTORS"
      2. "HEAP SUMMARY"
      3. "LEAK SUMMARY"
      4. "ERROR SUMMARY"

      Remember that each test may generate several valgrind sessions, each
      having its own summary. Be sure to review summaries of all valgrind
      sessions.

   2. After each test is completed, confirm that the status of the test
      itself is a PASS (i.e. that the test didn't fail).


*Expected result*

1. no memory leaks or memory errors were reported by valgrind.


*Actual result*

