#+TODO: TODO STARTED FAILED | DONE SKIPPED
# The vertical bar indicates which states are final states.

* DONE Tests for cwdaemon version 0.12.0
** DONE Debian GNU/Linux 11

OS: Debian GNU/Linux 11.7, x86_64
C compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110
libc: 2.31-13+deb11u6
libcw: 6.6.1 (unixcw 3.5.1-4)

*** DONE Build system tests
**** DONE make distcheck
*Steps*
Run "make distcheck" in main directory of unixcw.

*Expected result*
1. Distribution package is built correctly
2. Unit tests in distribution package pass.

*Actual result*
Pass

**** SKIPPED ./configure flags

Test that all flags supported by ./configure script produce "buildable" code.

*Steps*
Run ./qa/test_configure_flags.sh

*Expected result*

The script should not report any failure.

Example of failed test:

$ qa/test_configure_flags.sh
128: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-oss --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
Test of configuration flags FAILED
$ echo $?
255

Example of successful test:

$ qa/test_configure_flags.sh
128: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-oss --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
127: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-oss &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
126: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
[...]
3: ./configure --disable-oss &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
2: ./configure --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
1: ./configure &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
Test of configuration flags SUCCEEDED


*Notes*
The shell script may not work on systems without bash.

*Actual result*
Skipped - cwdaemon is not ready for the tests yet.


**** DONE regresion with make distcheck

Confirm that failure of 'make distcheck' after specific sequence of steps does not occur.

On Alpine 3.17 and with certain version of tests/unit_tests/Makefile.am, the
steps ended with compilation error, and distribution package was not created.

The error message was:

gcc  -g -O2 -Wall -Wextra -pedantic -Wswitch-default -Wimplicit-function-declaration -Wswitch-enum -Wfloat-equal -Wpointer-arith -Wcast-qual -Wwrite-strings -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wredundant-decls -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict -Wnull-dereference -Wjump-misses-init -Wdouble-promotion -Wshadow -Wformat=2 -std=c99   -o utils ../../src/utils-utils.o utils-utils.o  
/usr/lib/gcc/x86_64-alpine-linux-musl/12.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find utils-utils.o: No such file or directory
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:328: utils] Error 1
make[4]: Leaving directory '/home/acerion/cwdaemon/git_repo/cwdaemon-0.12.0/_build/sub/tests/unit_tests'



*Steps*
make distclean
autoreconf
automake
./configure
make
make check
make distcheck  # <---- In this step a compilation error was occurring


*Expected result*
Distribution archive is created correctly.

*Actual result*
Pass

*** DONE unit tests
Tests implemented in tests/unit_tests.

*Steps*
1. ./configure
2. make check

*Expected result*
All tests should be "PASS"

PASS: tests/unit_tests/utils
============================================================================
Testsuite summary for cwdaemon 0.12.0
============================================================================
# TOTAL: 1
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

*Actual result*
Pass

*** DONE functional tests

**** DONE Tests written in Perl

Execute each and every perl script in tests/ directory.

*Preconditions*
cwdaemon must be working, running on default port, with PulseAudio.
src/cwdaemon -n -x p

*Expected result*
All tests pass

*Actual result*
Pass

***** DONE tests/cwtest.pl

*Preconditions*
cwdaemon must be working, running on default port, with PulseAudio.

*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/cwtest_esc2.pl
*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/cwtest_esc3.pl
*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/cwtest_esc5.pl
*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/cwtest_esc64.pl
*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/cwtest_esc7.pl
*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/cwtest_esc8.pl
*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/cwtest_esca.pl
*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/cwtest_escc.pl
*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/cwtest_escd.pl

*Expected result*
Test passes

*Actual result*
Pass.

Bugs R0006 and R0007 have been added to README.dev.org. They don't have an
impact on end-user experience, so they will be fixed in future.


***** DONE tests/cwtest_escf.pl

For this test you may want to start cwdaemon with "info" log level, to see
information about switching of sound system:
./src/cwdaemon -n -x p -y i


*Expected result*
Test passes

*Actual result*
Pass



***** DONE tests/cwtest_escg.pl
*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/cwtest_short_space.pl
*Expected result*
Test passes

*Actual result*
Pass



**** DONE Tests written in C

Execute each and every binary in tests/test_00X*. Run each of them manually
to see the debugs printed in console.

*Preconditions*
1. Plug in USB/tty converter to USB socket.
2. ./configure --enable-functional-tests
3. make check (fix compiler error about files from unixcw)


*Expected result*
All tests pass

*Actual result*
Pass


***** DONE tests/test_001_basic_process_control
*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/test_002_reset_register_callback
*Expected result*
Test passes

*Actual result*
Pass


***** DONE tests/test_003_cwdevice_tty_line_options
*Expected result*
Test passes

*Actual result*
Pass

** DONE Alpine Linux 3.17
<Additional info about machine, OS, libs, compiler>
OS: Alpine Linux 3.17, x86_64
C compiler: gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924
libc: musl-1.2.3-r4
libcw: 7.0.0

*** DONE Build system tests
**** DONE make distcheck
*Steps*
Run "make distcheck" in main directory of unixcw.

*Expected result*
1. Distribution package is built correctly
2. Unit tests in distribution package pass.

*Actual result*
Pass

**** SKIPPED ./configure flags

Test that all flags supported by ./configure script produce "buildable" code.

*Steps*
Run ./qa/test_configure_flags.sh

*Expected result*

The script should not report any failure.

Example of failed test:

$ qa/test_configure_flags.sh
128: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-oss --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
Test of configuration flags FAILED
$ echo $?
255

Example of successful test:

$ qa/test_configure_flags.sh
128: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-oss --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
127: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-oss &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
126: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
[...]
3: ./configure --disable-oss &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
2: ./configure --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
1: ./configure &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
Test of configuration flags SUCCEEDED


*Notes*
The shell script may not work on systems without bash.

*Actual result*
Skipped. The package is not ready yet for this test.

**** DONE regresion with make distcheck

Confirm that failure of 'make distcheck' after specific sequence of steps does not occur.

On Alpine 3.17 and with certain version of tests/unit_tests/Makefile.am, the
steps ended with compilation error, and distribution package was not created.

The error message was:

gcc  -g -O2 -Wall -Wextra -pedantic -Wswitch-default -Wimplicit-function-declaration -Wswitch-enum -Wfloat-equal -Wpointer-arith -Wcast-qual -Wwrite-strings -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wredundant-decls -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict -Wnull-dereference -Wjump-misses-init -Wdouble-promotion -Wshadow -Wformat=2 -std=c99   -o utils ../../src/utils-utils.o utils-utils.o  
/usr/lib/gcc/x86_64-alpine-linux-musl/12.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find utils-utils.o: No such file or directory
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:328: utils] Error 1
make[4]: Leaving directory '/home/acerion/cwdaemon/git_repo/cwdaemon-0.12.0/_build/sub/tests/unit_tests'



*Steps*
make distclean
autoreconf
automake
./configure
make
make check
make distcheck  # <---- In this step a compilation error was occurring


*Expected result*
Distribution archive is created correctly.

*Actual result*
Pass

*** DONE unit tests
Tests implemented in tests/unit_tests.

*Steps*
1. ./configure
2. make check

*Expected result*
All tests should be "PASS"

PASS: tests/unit_tests/utils
============================================================================
Testsuite summary for cwdaemon 0.12.0
============================================================================
# TOTAL: 1
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

*Actual result*
Pass

*** DONE functional tests
**** DONE Tests written in Perl

Execute each and every perl script in tests/ directory.

*Preconditions*

cwdaemon must be started (automatically or manually), running on default
port, with PulseAudio.

./src/cwdaemon -n -x p

*Expected result*
All tests pass

*Actual result*

***** DONE tests/cwtest.pl

*Expected result*
Test passes

*Actual result*
Pass (2 cycles)

***** DONE tests/cwtest_esc2.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_esc3.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_esc5.pl

*Notes*
Start cwdamon with 'info' threshold for logs to see a log about exiting:

./src/cwdaemon -n -x p -y i

*Expected result*
1. These logs are visible in console:
  [INFO ] cwdaemon: escaped request: "<ESC>5"
  [INFO ] cwdaemon: requested exit of daemon
2. cwdaemon exits after receiving "<ESC>5" request.

*Actual result*
Pass

***** DONE tests/cwtest_esc64.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_esc7.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_esc8.pl

*Notes*

Make sure that user with which cwdaemon is running is in a group which has
"rw" permissions for /dev/ttyS0. The test checks different devices, and one
of devices should be a valid, accessible tty device. ttyS0 is such a device
(unless you plugged in an USB-to-serial converter and ttyUSB0 is also
available).

*Expected result*

1. cwdaemon doesn't crash when is asked to access different keying devices,
   including non-existent ones.
2. Test program passes.

*Actual result*

***** DONE tests/cwtest_esca.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_escc.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_escd.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_escf.pl

For this test you may want to start cwdaemon with "info" log level, to see
information about switching of sound system:
./src/cwdaemon -n -x p -y i

*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_escg.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_short_space.pl
*Expected result*
Test passes

*Actual result*
Pass

**** DONE Tests written in C

Execute each and every binary in tests/test_00X*. Run each of them manually
to see the debugs printed in console.

*Preconditions*
1. Plug in USB/tty converter to USB socket.
2. ./configure --enable-functional-tests
3. make check (fix compiler error about files from unixcw)


*Expected result*
All tests pass

*Actual result*
Pass

***** DONE tests/test_001_basic_process_control
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/test_002_reset_register_callback
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/test_003_cwdevice_tty_line_options
*Expected result*
Test passes

*Actual result*
Pass

** DONE FreeBSD 13.2-RELEASE
<Additional info about machine, OS, libs, compiler>
OS: FreeBSD 13.2 amd64
C compiler: gcc (FreeBSD Ports Collection) 12.2.0
libc: ??
libcw: 7.0.0

*** DONE Build system tests
**** DONE make distcheck
*Steps*
Run "gmake distcheck" in main directory of unixcw.

*Expected result*
1. Distribution package is built correctly
2. Unit tests in distribution package pass.

*Actual result*
Passed, but I had to use "gmake" instead of "make.

**** SKIPPED ./configure flags

Test that all flags supported by ./configure script produce "buildable" code.

*Steps*
Run ./qa/test_configure_flags.sh

*Expected result*

The script should not report any failure.

Example of failed test:

$ qa/test_configure_flags.sh
128: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-oss --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
Test of configuration flags FAILED
$ echo $?
255

Example of successful test:

$ qa/test_configure_flags.sh
128: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-oss --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
127: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-oss &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
126: ./configure --enable-dev --disable-xcwcp --disable-cwcp --disable-pulseaudio --disable-alsa --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
[...]
3: ./configure --disable-oss &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
2: ./configure --disable-console &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
1: ./configure &>/dev/null && make &>/dev/null && make check &>/dev/null && make clean &>/dev/null
Test of configuration flags SUCCEEDED


*Notes*
The shell script may not work on systems without bash.

*Actual result*
Skipped. The package is not ready yet for this test.

**** DONE regresion with make distcheck

Confirm that failure of 'make distcheck' after specific sequence of steps does not occur.

On Alpine 3.17 and with certain version of tests/unit_tests/Makefile.am, the
steps ended with compilation error, and distribution package was not created.

The error message was:

gcc  -g -O2 -Wall -Wextra -pedantic -Wswitch-default -Wimplicit-function-declaration -Wswitch-enum -Wfloat-equal -Wpointer-arith -Wcast-qual -Wwrite-strings -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -Wredundant-decls -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict -Wnull-dereference -Wjump-misses-init -Wdouble-promotion -Wshadow -Wformat=2 -std=c99   -o utils ../../src/utils-utils.o utils-utils.o  
/usr/lib/gcc/x86_64-alpine-linux-musl/12.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find utils-utils.o: No such file or directory
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:328: utils] Error 1
make[4]: Leaving directory '/home/acerion/cwdaemon/git_repo/cwdaemon-0.12.0/_build/sub/tests/unit_tests'



*Steps*
make distclean
autoreconf
automake
./configure
make
make check
make distcheck  # <---- In this step a compilation error was occurring


*Expected result*
Distribution archive is created correctly.

*Actual result*
Pass

*** DONE unit tests
Tests implemented in tests/unit_tests.

*Steps*
1. ./configure
2. make check

*Expected result*
All tests should be "PASS"

PASS: tests/unit_tests/utils
============================================================================
Testsuite summary for cwdaemon 0.12.0
============================================================================
# TOTAL: 1
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

*Actual result*
Pass

*** DONE functional tests
**** DONE Tests written in Perl


Execute each and every perl script in tests/ directory.

*Preconditions*

cwdaemon must be started (automatically or manually), running on default
port, with <platform's preferred sound system>.

./src/cwdaemon -n -x o

*Expected result*
All tests pass

*Actual result*
Pass

***** DONE tests/cwtest.pl

*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_esc2.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_esc3.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_esc5.pl

*Notes*
Start cwdamon with 'info' threshold for logs to see a log about exiting:

./src/cwdaemon -n -x p -y i

*Expected result*
1. These logs are visible in console:
  [INFO ] cwdaemon: escaped request: "<ESC>5"
  [INFO ] cwdaemon: requested exit of daemon
2. cwdaemon exits after receiving "<ESC>5" request.

*Actual result*
Pass

***** DONE tests/cwtest_esc64.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_esc7.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_esc8.pl

*Notes*

Make sure that user with which cwdaemon is running is in a group which has
"rw" permissions for /dev/ttyS0. The test checks different devices, and one
of devices should be a valid, accessible tty device. ttyS0 is such a device
(unless you plugged in an USB-to-serial converter and ttyUSB0 is also
available).

*Expected result*

1. cwdaemon doesn't crash when is asked to access different keying devices,
   including non-existent ones.
2. Test program passes.

*Actual result*
Pass

***** DONE tests/cwtest_esca.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_escc.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_escd.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_escf.pl

For this test you may want to start cwdaemon with "info" log level, to see
information about switching of sound system:
./src/cwdaemon -n -x p -y i

*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_escg.pl
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/cwtest_short_space.pl
*Expected result*
Test passes

*Actual result*
Pass

**** DONE Tests written in C

Execute each and every binary in tests/test_00X*. Run each of them manually
to see the debugs printed in console.

*Preconditions*
1. Plug in USB/tty converter to USB socket.
2. ./configure --enable-functional-tests
3. make check (fix compiler error about files from unixcw)


*Expected result*
All tests pass

*Actual result*
Pass

***** DONE tests/test_001_basic_process_control
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/test_002_reset_register_callback
*Expected result*
Test passes

*Actual result*
Pass

***** DONE tests/test_003_cwdevice_tty_line_options
*Expected result*
Test passes

*Actual result*
Pass
