Demo downtime (August 25)

For the record, the machine panic’ed. The vagrant development virtual machine went down with:

192.168.121.108 - - [24/Aug/2017 13:12:50] "GET /static/i/font-awesome/fa-arrow-circle-o-right-white.png HTTP/1.0" 200 -
192.168.121.108 - - [24/Aug/2017 13:12:50] "GET /static/i/font-awesome/fa-arrow-circle-o-right-blue.png HTTP/1.0" 200 -
Traceback (most recent call last):
  File "source.py", line 450, in 
    app.run(debug=debug, host='0.0.0.0', port=8080)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 841, in run
    run_simple(host, port, self, **options)
  File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 737, in run_simple
    reloader_type)
  File "/usr/local/lib/python2.7/dist-packages/werkzeug/_reloader.py", line 265, in run_with_reloader
    sys.exit(reloader.restart_with_reloader())
  File "/usr/local/lib/python2.7/dist-packages/werkzeug/_reloader.py", line 124, in restart_with_reloader
    close_fds=False)
  File "/usr/lib/python2.7/subprocess.py", line 522, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 1376, in wait
    pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
  File "/usr/lib/python2.7/subprocess.py", line 476, in _eintr_retry_call
    return func(*args)
OSError: [Errno 14] Bad address

but the panic only happened when I tried to vagrant up development. I suspect something wen wrong on the underlying host because there was:

Aug 24 14:38:20 demo kernel: [1463166.211526] INFO: task nginx:26644 blocked for more than 120 seconds.
Aug 24 14:38:20 demo kernel: [1463166.212829]       Not tainted 4.10.0-24-generic #28-Ubuntu
Aug 24 14:38:20 demo kernel: [1463166.213602] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 24 14:38:20 demo kernel: [1463166.214681] nginx           D    0 26644  26641 0x00000000
Aug 24 14:38:20 demo kernel: [1463166.214684] Call Trace:
Aug 24 14:38:20 demo kernel: [1463166.214730]  __schedule+0x233/0x6f0
Aug 24 14:38:20 demo kernel: [1463166.214731]  schedule+0x36/0x80
Aug 24 14:38:20 demo kernel: [1463166.214747]  kvm_async_pf_task_wait+0x1c9/0x210
Aug 24 14:38:20 demo kernel: [1463166.214750]  do_async_page_fault+0x5f/0x70
Aug 24 14:38:20 demo kernel: [1463166.214755]  async_page_fault+0x28/0x30
Aug 24 14:38:20 demo kernel: [1463166.214759] RIP: 0033:0x7efc5d988804
Aug 24 14:38:20 demo kernel: [1463166.214763] RSP: 002b:00007ffc46513cc8 EFLAGS: 00010202
Aug 24 14:38:20 demo kernel: [1463166.214764] RAX: 00007ffc46513e35 RBX: 00007ffc46514620 RCX: 0000000000000005
Aug 24 14:38:20 demo kernel: [1463166.214765] RDX: 0000000000000005 RSI: 000055e795964f5e RDI: 00007ffc46513e35
Aug 24 14:38:20 demo kernel: [1463166.214765] RBP: 00007ffc46513d00 R08: 0000000000000000 R09: 0000000000000000
Aug 24 14:38:20 demo kernel: [1463166.214766] R10: ffffffffffffffff R11: 0000000000000000 R12: 000055e795957836
Aug 24 14:38:20 demo kernel: [1463166.214766] R13: 0000000000000005 R14: 000055e795957c60 R15: 000055e796653880
Aug 24 14:40:21 demo kernel: [1463287.044467] INFO: task nginx:26644 blocked for more than 120 seconds.
Aug 24 14:40:21 demo kernel: [1463287.046374]       Not tainted 4.10.0-24-generic #28-Ubuntu

In /var/log/syslog. Before that syslog is clean from 07:00 to 14:40. I hard rebooted from the horizon panel.

The SecureDrop demo category was created in the forum and linked from the demo page so it is easier for people who run into a problem to report it or just discuss the demo in general.