website/content.org

#+hugo_base_dir: site
#+hugo_section: posts
#+STARTUP: logdone
#+author:  Valentin Boettcher

* Pages
** About
:PROPERTIES:
:EXPORT_HUGO_SECTION: /
:EXPORT_FILE_NAME: about
:END:

This is the web presence of Valentin Boettcher a.k.a. Hiro98.

Valentin:
  - currently studies physics in Dresden.
  - codes stuff.
  - is a science-fiction junky and space enthusiast.
  - loves jamming with the saxophone.
  - is hooked on LISP and Clojure.
  - appreciates C++.
  - likes his [[https://blog.splitkb.com/blog/introducing-the-kyria][Kyria]].

You can check out my [[https://github.com/vale981][Github]], [[https://gitlab.com/vale9811/][Gitlab]] and [[https://invent.kde.org/vboettcher/][KDE Invent]] profiles for most
of my projects as I publish the source to most things I produce.

I've got a [[file:static/docs/cv_ger.pdf][CV]] in German.
<!--more-->

My usual internet/user/nick name is /Hiro/ or /Hiro98/ after Hiro
Protagonist from Neal Stephenson's novel "[[https://en.wikipedia.org/wiki/Snow_Crash][Snow Crash]]", who is usually
depicted in profile pictures like the following one.

#+DOWNLOADED: screenshot @ 2021-08-01 15:14:54
#+CAPTION: By [[http://www.benzilla.com/?p=4209][Ben Towle]].
[[file:static/images/Pages/2021-08-01_15-14-54_screenshot.png]]

*** Things Made
- The source of this website lives [[https://github.com/vale981/website][here]].
- I've written a bachelor's thesis about [[https://git.io/JBPZg][Monte Carlo Event Generators]].
  I even built my own in python.
- A simple, easy-to-use and firewall penetrating [[https://gitlab.com/vale9811/doccam-pi][control software]] for
  streaming stuff from ~rtmp~ cams to youtube.
- Along with a commilitone I've written some lecture notes about
  [[https://gitlab.hrz.tu-chemnitz.de/strunz/skript-quanteninformation][Quantum Information]].
- The website of [[https://simunova.com/][SimuNova]].
- The software behind [[https://klausurnoug.at][klausurnoug.at]].
- A [[https://protagon.space/stuff/neutrino_oscillations/][visualization of neutrino oscillations]].
- A [[https://github.com/vale981/SecondaryValue][python library]] to calculate the gaussian error propagation.
- A [[https://git.io/JBPZX][small hack]] to scrape and organize physics courses at the
  TU-Dresden. [[https://protagon.space/stuff/vertiefungs_scraper/][See it in action]].
- A script [[https://github.com/vale981/wunderlist-to-org][to convert the wunderlist export to org-mode.]]
- A script to create an arch linux image with [[https://github.com/vale981/archiso-bcachefs][bcachefs]] support.
- I've made the [[https://sherpa-team.gitlab.io/][website]] and some of the continuous integration for the
  SHERPA project.

*** Activities
- I am the admin and the most enthusiastic user of the [[https://physik.protagon.space][TU-Dresden
  Physics Forum]].
- I am an occasional contributor to [[https://invent.kde.org/education/kstars][KStars]]. As of late I have
  participated in the Google Summer of Code to implement a better
  Deep-Sky-Object backend.

  I also implemented and now maintain the [[https://invent.kde.org/vboettcher/kstars-catalogs][KStars DSO Catalog
  Repository]].
- Sometimes I release some noises on [[https://soundcloud.com/the_dj_c][Soundcloud]] and [[https://afa-music.bandcamp.com/][Bandcamp]] under the
  name "Abschreibung für Abnutzung".
- I am also the maintainer of the [[https://www.doc.govt.nz/nature/][RolaCam]] backend which is also used
  by the [[https://www.urbanwildlifetrust.org/portfolio/live-cam/][Urban Wildlife Trust]] for their [[https://www.youtube.com/channel/UCLizlM6gpaVHTKPo7spoqlA][Wildcams]].
** Contact
:PROPERTIES:
:EXPORT_HUGO_SECTION: /
:EXPORT_FILE_NAME: contact
:END:
Just [mail](mailto:valentin@boettcher.cf) me. Spambots not welcome :P.
I'm on [dev.to](https://dev.to/hiro98).

Here is my ~GPG~ public key.
#+begin_src
-----BEGIN PGP PUBLIC KEY BLOCK-----

mQENBFsw8ywBCADP+YJLyntBCNgld21x2FG811w2VFLizjrYRakgIwqSCEYyeUd0
jXhVi8SzsPEQlwPEUtDgLQekRvsDXhT2mW4jq5g46qX1OSrzAh3H1R0e9dAVNC59
fNGDOFVEKmNdPhSOprwTMHQLsebo13gc5xm5SDSGx780xPGvRHMkPyQdQw6mufTZ
7ejbkAnsHoZKANhfu68O6GN71Rpogp4u6o/ac4TpzBNb+zuNL2NCckkf3oT9Lokb
r/I0KsV0xAlbbmth4vMFeKmhlgR/TYWNqNe+Lq9tIbyX780gAzEItdgRJFT6LEjd
veiAe+iWw/EJPZy9+qenoCGns//zagov1iMFABEBAAG0NVZhbGVudGluIEJvZXR0
Y2hlciAoTWFpbiBFbWFpbCkgPGhpcm9AcHJvdGFnb24uc3BhY2U+iQFOBBMBCAA4
AhsDBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAFiEERamZF1eM1imfX7W0wi1N5Nez
LRkFAl5mg1IACgkQwi1N5NezLRnLOAf/RRUjZzVOu+NMShk2G+fEodNTem8pD5r1
Up+qEdywyPlV87UtK2y2Q4CB4mwuqJV7OwgQuCMVOJhSGstpQ6bQcl+VZ1p9dMzr
TVqcaHWFexLp0+h/S6loGGySh0txvzYkTLfAih3osXmQvvwNjw8BggT7wWztS9q2
cmvZ1sT3TP5Vq67WyBW6We9a6R5hw9lb+IOQct2dJkskxhI6CHyZdKXvxUMk0mE4
2uf9+lbFzBTJqs+VT5TfLnTcA6Vhu6ke50mSCuMkwKEjER/lmNCx6TuSYxuVGczi
hDfs4RDzpewMIkkoZK6YBtKvyIcyY5lBzmmgNJmmVbmCgFmQiEgN1IkBVAQTAQgA
PgIbAwULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBEWpmRdXjNYpn1+1tMItTeTX
sy0ZBQJg1fgxBQkJZ2wAAAoJEMItTeTXsy0Zgk4H/1BVJKhLLwnj5iWSFk+2OliU
DmFhkWjNWZpOCvaBSxNQNkod+9jjQ+Rg0+thRyLm6oZOg4g/iluslds1Hxf++pxz
/HJdJ7/oQ/BG3RZrh2Lc2BeptVntFoGR/K0VoJYbwjOXP8B8lnGJZEX6/QnT9Fkq
ZFrm+8CgYhKradeTl30LYA5TBCT+BrtzervHv6CrmN6WcROdwYHj+RHBXauswG8k
eeoDVXLpg1XiMcWB5dhiVQNxcoAiOL9k5Z6xSR6Zgd5ediXBvbL3k3nsu/ZtWeR3
7dLFnba2zi2IGDL48fchgFSWWM8SemKeQ9wiPb+no+uEFDWmwa/+cAm54DFeJ420
KlZhbGVudGluIEJvZXR0Y2hlciA8dmFsZW50aW5AYm9ldHRjaGVyLmNmPokBTgQT
AQgAOAIbAwULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBEWpmRdXjNYpn1+1tMIt
TeTXsy0ZBQJeZoNSAAoJEMItTeTXsy0ZYv8H/jdOvmnXV2Og2vEKJsoPDgvfzqww
EopE8j6CADBdyKt+51CgGkccCj7SzCG80qUMNJAeKU4c5nN7OVGJ+b3hNd2G0f8j
V5mRlXNIl6m+7MmflUph9PKMBRlezUTA0enBIJIvxe5l8M2dp4zO09HGtoexiVcg
FX3u4DsKy2BT8c3scfKcEQXFQV+zKLBm4ko43LSEEtTS2JOCyTNorTkS6b0JBN6N
Opmbb0hnx45+731vuz0/F3OsLbPJ4objiLVoM1RaY0tx0z4Kb5J9d+9y0eHAtxt/
MhemqbNXPsFZFHW2mLxEtK+dZuCeCyh+qBcFUWvPCkXk0X2bPcBLJwabJYKJAVQE
EwEIAD4CGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AWIQRFqZkXV4zWKZ9ftbTC
LU3k17MtGQUCYNX4LAUJCWdsAAAKCRDCLU3k17MtGe0eB/4rIF+j316MEdBE1/sg
8yNMM/lnYh3BBtF0WmR0NVZA5IFMJXZg029fqRtkcUSkro21XIXkMkmz5hmG2GHt
9i6/Or/irfhwZKLPBeN7KT7TJkLUNP6V2dmIF4Juz0+KIE2aBFg3zDLA6CrTCDig
I7GE/8adb40SXC0XLOdLnxO0SUr1VL2Hyi1Jw6AgroYynTbAzWpYP+JkSPW1voUl
/TfpPxMvgRsJS10XTQUNdYu54r/ByM/tYb7f4+KLaUNTW8zlViS8niNaASRe3e/2
5Qn/9aeytRlzZUi3UZ+tb5TrTlcNdpER16GPcb63ABfedADyZXd8NNafLDuRf/bF
KR5juQENBFsw8ywBCAC4LcWjxCONetL5q33xGMLJbGU49FkMT1qpwiLTgBTG3jy+
/0TbNYnC4iDulUm0yardjPHmYLwswnzzzYWSvbpE+cKJdULw37+R0f+28eyN53dY
+XZ6yb6KiFAjQCwcSaLmB7a4YIEOqpz6ake7qiuhJ6Lx5hEfp42o4kR2QxugiMqe
Z+8IwhfLM+0AlP+rqdJdkxvw+dXW0C/F9hZcRAgRKZ1N+eQU8uUME5QXI8grhtd9
ThLFCo3dl6a+dI/TkRpYLVR9c4obcocUtTnzOl4L/o5FedDq020D9+MdXqbRQVTs
dF2/jTVbBwhDcO6Vi46OrcFvTQ7d+TaOhuhsshpVABEBAAGJATwEGAEIACYWIQRF
qZkXV4zWKZ9ftbTCLU3k17MtGQUCWzDzLAIbDAUJA8JnAAAKCRDCLU3k17MtGVe/
CAC+lp2MhdN7Enr+RE8qDz0s4090uDVI8mQWi663NRCd1uUHrQTju1dqA0MEjYFG
AVhIpnHcjroWMyGNFQphs7ekLmBqsHMf5PYOtGdqKbI+y0TCfQ+GXF9tUK9a8Xp3
EwLHgGA0SaeyEZj9TwmsoetcYtZA3FNiCy2QEzsNL0dpytZFPktvR3I52eE3cxYw
9gI9x9/Z+n0vyQGL1o1CsB/z2Vh+g8QfINHHfnX1PQp3Jjbg/uhjw5mZpn6+i7/k
UC9VICpHoT3U2xQHrDjCol8nvH4PSfpyPMr1is7S5I05Q1xja2GQgt5qmtpGyx4K
AKTcpWc4R+XgmwwIF5NWaadPiQE8BBgBCAAmAhsMFiEERamZF1eM1imfX7W0wi1N
5NezLRkFAmDV+E0FCQlnbCEACgkQwi1N5NezLRmRQAgAuItAuBZYFUQb1cOxeKQY
V3wETwd9TduZUAEUQzZQhNcfoOJhAVnOrKqUpvP11cVjxXD2ox8GaPi4NXl5BPa9
EEfnRUJQ28R9JQmDNqWrCcYjNGtuw1JwnsJKanSgV+L/8MJF5DxfqaiAKfq+t0By
3a/7SRa0X6LxciLn/uoqdy49/L223oe2InC3PbiJz3RnIOZGLTvCeYEjumZcfUx+
iM4Kzj7zD0tbH9fvboexyacp8034qlD48LE9/N9rWkvLYUfm0DiIkyt8wCUHB1r+
PQw8+Q2mSJ4JBksco0BsL96cH7LhL9RBB+UkNjxiBYaPY2vxhLImWPTCY39yA4wM
I7kBDQRfGCGvAQgAsCHAvggPmnwaKV4lWhAm8+dI92iYzZfkNX54jy92UceFG7p2
9bQOnlG2SMWilIMTI9X9+oeaDx41mlAiTQrWnQ5udum3x3UYCAoQXhRZsxFjO1ob
1Ycc0K0GT0RuQrggFay7s39hpTGI5d8lYBjcLJ3oepUCe4nEvIn5rhvypbUiRBo/
+G0g9ysKKT9ehDDdp0qMxM8kRJi215jjctxFSGyvXerbb77rMkKYeVp0U58SA7WY
9yKv4tn1nBYVaTNPMwHwt17MrIz5fSZjIexl9lUpoHGEDNu+6QPdSkfhrc99Km2N
3wC5lNDzyjYifox6kzEtGmMrmIHjDiIysA9N7QARAQABiQE2BBgBCAAgFiEERamZ
F1eM1imfX7W0wi1N5NezLRkFAl8YIa8CGwwACgkQwi1N5NezLRl1vQgAiES+jtp7
+bpW+rdUNGbCJAS28gQpn9Bkt54c+WyDEZbMST1cJgyuLX/0Zf8xu2Bwwxb2LYSk
J7N83gDsXie0zmD3NoHE27FPEGIsVHoo0qcgcg8MY66O6ZRTr/njMF+DGPqV7FtM
x5x9qfM7YMZ+3S49DmGgHRBAELGuyqfvLiTM5unlnDDA9N755VCHZ5optOcuguMY
y4IhPexft/6cRBfLPH2EqpoDVuKcQDr0Y1OKinsgmzSB8qUYt6SqWTN51WfhQMtz
Ejh3MC58ExFWmgOE2cS++8vk7ntprYEeeSesoXPXvXW8AvfTrjnapIg2kjtuDRBH
e6q7By8xuUSSOrkDLgRfvmUSEQgAjNMfeXd6jy310NXYQqw9zcE71ubaLs7drlmJ
lrp+vEDfAq351BpbDtE/AVWrMujccFVx/fDJQoipXi+8dsJswZT152NNXfEceeaU
56CUKXvlOfmDjqeSehI6/L9g7sXH8hSpngyx6CmBViP4MQPwwlwrb+N52l8ub1tn
nJJf7sgBO+SnRpKa5ZRHx6YHHsQ5vYwzowwR6iK0+6NojNzlVp+8F5qdVPvygEF4
eOz1e2dmKgMEjzvaf/Z64WYGQeKsFXreRjskOuMMmDjJ/6sduOjDvrHZW04jRNFZ
rwZbaE+iVv1LMxTxV5lus35lum/UUuShH6EI/fWAs/8HLe8IBwEAulBzs2L/oLw/
tw1gjL1D2PUDzaTaNl0LpSVzdB5CbNEH/idU1SSvv8Vf7uVnAnxJXQUuLnDuz3Pq
uym4TvoWRM/OKd/uj+6d+BeoriUDFFuRhNJcvNLyDfU/9LOZMbxjQer+MYc/aNsd
PUUlJeSgkc7pGdToC2/2IqTntb+ZYRUwUDAGMElxTARIMNcASaIgVcLwkwaeX4aN
DzOxvRyh47MINVXUMfqEuDXZYEQdVXVwQQQQl3cD4q6WEiZLOhmyzIq4DTyTA/1B
OExM/HvJgeczIOtGMfO3ZyLYQ7KpK1/1ZWMVV7tl1ZoXUL4JdgORqNIrbY3qqEQu
riPp0m4ypKqmj7W0FtYut5TDu72O8w4hb4+3W1qMjWIrW32h0KuLHT4H/05fwzpc
e56djf88QTt8C4CUx6Anks/+RmPtyNFHylQ6mgX1CfntrD4Mk4N+qRP9iZRHZuxR
AkzVcIHs2p/FX5/83pyR5a3krsJvVxoeER5VhoIzZedRvwqGiAn6sA5uSPa9IIBc
6oefAsjleSHZlwC5tQ3zdtFidI3xnV65myppv5uMo73GZFM28/RmSCJgnAX6QxfL
/EAXlgxa7gVWbJw6b+QAMPvH58+gm8z3rN9CXIb6k6GBKoXcnEH7imTXQ0ZhJRA5
DWu0FCziVaJ7WgXdeGRHSjd7AqAaDqUWi8DseFfBVpCX9r6nMJY1WRv8iLIb3d+U
7LS3EBh18M9PBWmJAa0EGAEIACAWIQRFqZkXV4zWKZ9ftbTCLU3k17MtGQUCX75l
EgIbAgCBCRDCLU3k17MtGXYgBBkRCAAdFiEEOsdCeTthkZeWIuEa4DThK3r1as4F
Al++ZRIACgkQ4DThK3r1as5azAD+KqMX5KcfcL4ibM0zpCUoC8ncFwhC1rgf263F
uqAt08YA/iOIUT9TsoW4Y+e/Zx9jpBS5w8byg5tPfGusiaLG35TO05UH/2vYKVRQ
URFIsxpllAIE0fCDkgVTrACYfW1GXVF2Z+2JZ9iRXw//zK3HXTPTKjlmkob+EK+5
AmrRoGuN9y4L5FEpLuycXO6u6KAQD+0fN0GUPFxzwf8kUSfZbaRWIJKEsDd/NSgJ
Cu4UMICGz5Q6kzkBtRkePlhgrS6+PeJHJ0C6P1T/qVwvI/eIcPRqgeJOndG9BtsV
5/QO/rUXtMoLUVHoFANr3JNuFGZpZ0xnQEd2QGAxOU4EfQ0djt1a/YNp3qgBbU3f
z9SvuganEEyWovdM1ojmJxkU7jPa6SztIaIUKsl6Zn7d8R7fJJSP3po38yuGcF+m
4gcGTSpBB5xsqnc=
=x6uy
-----END PGP PUBLIC KEY BLOCK-----
#+end_src
** Impressum
:PROPERTIES:
:EXPORT_HUGO_SECTION: /
:EXPORT_FILE_NAME: DSGVO
:END:

Verantwortlich für den Inhalt nach § 55 Abs. 2 RStV:

#+begin_src
Valentin Boettcher
Raecknitzhoehe 15
01217 DD

Kontakt:
E-Mail: hiro at protagon dot space
#+end_src

* Blog
** KDE                                                                :@KDE:
*** KDE GSOC: Intro                                                  :GSOC:
:PROPERTIES:
:EXPORT_FILE_NAME: gsoc_intro
:EXPORT_DATE: [2021-06-27 15:00]
:END:

Hi folks, talking to you over the interwebs is Valentin Boettcher who
is overhauling the Deep Sky Object (DSO) system in the KStars Desktop
Planetarium for the Google Summer of Code anno domini 2021.

This is the first post in a series and rather late in the coming, so
let's get right to it.

I'm currently studying for a master’s degree in physics at the TU-Dresden
in, you've guessed it correctly, the beautiful city of Dresden
(Germany). In Germany, we do have two study terms per year and the
summer term usually coincides neatly with the GSOC so that I couldn't
participate in past years. This time around however, my schedule was
finally sparse enough for me to have a go at it, and here we are :).

My first contact with KStars development was back in 2017 while I
spent a year in New Zealand and had a lot of time at hand. My
reasoning was, that I could learn mathematics and physics in UNI and
should funnel my enthusiasm into familiarizing myself with software
development and the open source software community. I promptly wiped
my hackintosh laptop to put Linux with KDE on it[^3]. After reading
ESR's famous ["How To Become A
Hacker"](http://www.catb.org/~esr/faqs/hacker-howto.html), I followed
the advice given therein, which was to find an open source project and
start hacking on it. I already liked KDE and space, so KStars was in
the center of the Venn-diagram :P.  I went ahead and busied myself
with one of the junior jobs listed on the KStars web-site[^2]. I
quickly found that I liked figuring out how stuff in KStars worked and
also got in contact with my mentor Jasem Mutlaq who was always
available to answer questions and endure my barrage of instant
messages on matrix :P. My second job was to draw comets a tail and
learned that it is wise to do some code archaeology before going ahead
and implementing functionality that is already present. From there on
I contributed more or less regularly when I found the time in my
semester breaks.

Now, finally, let's talk a wee bit about the actual GSOC project.  In
KStars, everything that isn't a Star or an object in our solar system,
an asteroid, a satellite or a comet (I'm sure I forgot something) is a
deep sky object (DSO). Prominent members of the DSO caste are galaxies
(think M31, Andromeda), asterisms and nebulae. Of course there are a
plethora of catalogs for specific types of DSOs (for example, Lynds
Catalog of Dark Nebulae) as well as compilations like the New General
Catalogue.  The system for handling those catalogs in KStars has grown
rather "organically" and is now a tangle between databases, CSV files
and special case implementations. Many catalogs were mentioned
explicitly in the code, making it hard to extend and generalize. Also,
the sources of the catalogs and methods how they were transformed into
the KStars format were inhomogeneous and hard to reproduce, making
deduplication almost impossible. Finally, KStars just loaded all the
DSOs into memory and computed their position on the virtual sky for
every draw cycle, which made all too large catalogs infeasible.  My
task is now (and has been since the beginning of June) to implement a
unified catalog format which can be loaded into a central database and
supports deduplication. Furthermore, taking inspiration from the
handling of star catalogs in KStars, the objects should be trixel[^1]
indexed and cached in and out of memory (but only for large
catalogs). Finally, it would be very desirable to make the
creation/compilation of the catalogs reproducible and easily
extendable to facilitate future maintenance.

This sounds like a big heap of stuff to get done and in the next post
I will be detailing how it's going so far :).

Cheers,
Valentin


[^1]: In KStars the sky is subdivided into triangular pixels "Trixels".

    Assigning each object to a trixel makes it efficient to retrieve all objects from a certain part of the sky.

[^2]: which had to do with figuring out why some faint asteroids where missing

[^3]: which I knew from my school time when I used it on my netbook because there was a cool neon "Hacker" theme for it :P

*** KDE GSOC: Community Bonding and First Coding Period (May 17 - July 11) :GSOC:
:PROPERTIES:
:EXPORT_FILE_NAME: gsoc_1
:EXPORT_DATE: [2021-07-11 15:00]
:END:

Of course the task I described in the [[*KDE GSOC: Intro][last post]] looks and is quite
monumental. That is why I laid some of the groundwork for my GSOC
beforehand (in the actual German semester breaks). This work continued
in the community bonding and first coding period and will therefore be
described here.

But first I want to thank my mentor Jasem Mutlaq for his support, his
patience with me and his nerves of steel. My mood levels were somewhat
similar to a huge-amplitude sine wave those last weeks.

Now to the meat...

I began by studying the existing deep sky object implementation in
KStars to identify what structure the new catalogs should have and
what the smallest irreducible core of functionality was I could
replace to make integration easier. I discovered that the catalogs
were a mix of SQL databases and text files, somehow loaded at startup
and then appended to some linked list. There was some deduplication
implemented but like most DSO code it was oddly catalog
specific. Especially the Messier, IC and NGC catalogs were often
mentioned in the code. Also the explicit distinction between stars and
DSOs made writing general code complicated but I found a consistent
set of data fields shared by all catalog objects which all admitted
sane defaults. It wasn't bad code or anything like that. Just the
product of "organic groth" with many thing I wanted already present in
some way but somewhat all over the place. I admit that I studied the
code just enough to find out what exactly I had to replace and maybe I
could have reused more of the existing code but I've picked this
specific path in the multiverse, so let's get on with it. Just a shout
out to all who did previous work on the DSO code among whom are, just to
name a few, Jason Harris, Rishab Arora, Thomas Kabelmann and Akarsh
Simha.

With this knowledge I was able to go forward and devise a concrete
plan for implementing the new DSO system. First of all, albeit I would
love to use ~std::variant~ and some kind of entity component system
for the different DSO types I settled with a one-for-all type for deep
sky objects. The primary reason for this was, that KStars uses ~C++14~
which lacks variants (and the extremely useful
~std::optional~). Furthermore the DSOs all share common structure, so
this was just the simpler and thus preferable option. The second
design decision was not to load all of the DSOs into memory, but
instead to take inspiration from the deep star catalogs. For one they
are dynamically loaded from a special trixel indexed format so this
already was within the formulated goals of the endeavor. On the other
hand the notion of having "canonical" copies of catalog objects in
memory and syncing their mutation with the database system seemed
overly complicated. The catalog database should be the single source
of truth and not the (ephemeral) memory of KStars.

When a specific object is needed, it should just be retrieved from the
database locally in the code instead of searching some in memory list
in KStars or shooting around with pointers. This notion is somewhat at
odds with how things were and are done in KStars which created some
interesting problems later on as we shall see. These ideas somewhat
dictated the rest of the plan which I (for the first time in my
programming career) completely wrote down in advance. The heart of it
all is the database manager which abstracts maintaining, reading from
and writing to the database. As always one should justify the creation
of a special data type. In this case it was the requirement that the
database access should be painless and could be handled locally
anywhere in the code just by creating another instance of the database
manager. The manager should handle retrieving objects and catalog meta
information as well as importing, editing and exporting catalogs.

The structure of the database itself was another point of
consideration. Naturally each catalog should have its own table. But
how should deduplication work?  The method I settled on is really
quite simple. Each object gets a (relatively stable) hash that is
calculated from some of its properties which is henceforth called the
ID.  When two objects (from different catalogs or otherwise) are the
same _physical_ object, then they will both be assigned the same
object id (OID) which is just the ID of the object in the "oldest"
catalog (with the lowest catalog id), trying to make it stable under
the introduction of new catalogs. Additionally each catalog is
assigned a priority value which is just a real number (conventionally
between zero and one). When loading objects from the database into
KStars and there are multiple objects with the same OID only the one
from the catalog with the highest priority will be loaded. This simple
mechanism should cover the requirements of KStars quite well and is
relatively easy to implement.

There I ran into an issue that demanded some research and
table in the database.  The simplest option would be just to create a
benchmarking. Remember that each catalog is represented by its own
so-called view, a dynamic "virtual" table that combines all the
catalog into one homogeneous table. SQL magic could automatically
perform the deduplication algorithm outline above and everything would
be fine and dandy. However, benchmarking revealed that actually
writing the view into its own table, henceforth called the master
catalog/table, increased the performance quite considerably, enough so
to justify the increased complexity in the implementation. And then I
discovered SQL indexes. A gift from the heavens! They increased
performance on loading objects in a trixel from the master catalog
roughly threefold and I was sold on the master catalog approach. So to
summarize it all; a deduplicated view of the combined catalog tables
is being created and then written into the master table. This has to
be done for every modification of the catalogs but is relatively fast
(just not fast enough to be done 20 times per second). Later
experimentation showed that this approach could accommodate catalogs
up to a million objects in size.

I also created a catalog file format, which is just an sqlite database
file with the application id set to a special value with almost the
same structure as the catalog database proper. The application id
enables KStars to check if the database is really a catalog file and
not to rely just on the structure of the contained database for
that. In the future the ~file~ command and other utilities like file
managers could be made aware of this special application id to
recognize the catalog files. We will leave it this level of detail for
now. For more details please refer to my [[https://protagon.space/stuff/kstars_cleaned.org][notes]].

Of course the operations on catalogs have to somehow be accessible in
the GUI of KStars so this was another point of action. Before that
however the glue between the database manager and the usual sky
composite system had to be implemented. In KStars different types of
objects (Stars, Comets, Asteroids, etc. pp.) all are implemented as
components with a unified interface. These components provide methods
for loading and drawing objects, as well as utilities to find objects
near a certain point on the sky and similar things. The loading and
drawing part was relatively simple to implement. The drawing code
could be straight up reused from the old implementation and the
loading was essentially covered by the database manager but with a
twist.

To support very large catalogs it would be desirable to only have
objects in memory which are currently visible. Thus a LRU cache was
implemented with the trixel id, which essentially labels a portion of
the sky, as key. This cache is fully unit tested and relies completely
on standard library containers so not a single pointer appears in the
code.[fn:1] As an added bonus, the cache is completely transparent by
default and only takes effect if configured to so and therefore
includes the typical use case of comparatively small catalogs up to
ten-thousands of objects.

But here the culture clash between the new DSO implementation and the
traditional KStars way of things became apparent. In many places
KStars expects pointers to so called ~SkyObjects~ with no real clue as
to where they are actually stored and how their memory is managed and
with the implication that the object is expected to live
forever. Well, the DSOs from the catalogs aren't supposed to be kept
around forever and thus a compromise is in order. So whenever a
pointer to an object is required, it is inserted into a linked
list[fn:2] in a hash table with the trixel as index or is taken from
there if it's already present. I hope that we can eventually
transition away from raw pointers and manage life time either
explicitly or with smart pointers.

With this done and basic drawing working I went on to implement a
basic GUI for catalog management[fn:3].  I also wrote unit tests for
the database functionality which proved itself as very useful later
on. After that I couldn't delay anymore. Back when I implemented the
component for the new DSOs I went as far as getting it to compile and
not much further[fn:4].

Now I had to go around and find out what broke. A lot broke and I did
not find all of it until the big merge :P.  A rather interesting
source of work happened to be the way metadata like observation notes
and image links were stored. They came from a text file and then were
loaded into the sky objects at startup and somehow synchronized with
the text files on mutation. This, of course, played not well with the
new DSOs as they were ephemeral. So I replaced the whole shebang with
a hash table which incidentally improved startup performance. The rest
of the integration work was similarly interesting and continues
today. I will not go into it further but feel free to look at the
KStars commit history.

Just yesterday I added a feature back in that I had axed accidentally
to the dismay of its original author. That showed me that I am not
entitled to judge the merit of individual features and whether they
could be sacrificed for the "greater good". The answer is: They
cannot! Another lesson I've learned is, that too much magic just ain't
no good. I had created a variadic template wrapper for the ~QSqlQuery~
type for syntactical convenience and shot myself in the food with
it. It ended up obscuring an error message and prevented me from
reproducing a crash that users on certain platforms were
experiencing. After a not-so-great couple of days I, with the help of
two kind people, finally found the lowest common denominator of the
problem: an old, but still supported version of QT which bundled an
old version of sqlite which in turn did not support the ~NULLS FIRST~
directive that I was using. Turtles all the way down. Although I
tested all my changes on KDE Neon (I am on NixOS primarily) the wise
thing would have been to develop or at least test everything with an
older QT version from the get go.

Also, although I had put in version checking into the database code, I
didn't provide a mechanism for upgrading the database format to new
versions. This I now remedied by introducing a simple mechanism that
applies database modifications successively for each version
upgrade. So if I go from version two to version four it will be
upgraded from version two to three and then to four which I understand
is the way those things are usually done.

Now, I did do at least some "constructive" work, adding a (admittedly
ugly) CSV importer so that users can import arbitrary CSV-ish
catalogs. The greater chunk however I will cover next week: The python
catalog package tooling with continuous integration and
deduplication. The catalogs churned out by that framework are then
installed via the ~KNewStuff~ framework. I discovered two interesting
bugs in this framework because KStars seems to be almost the only
program using the framework in this specific way.

If you made it this far, I applaud and thank you for your endurance.
See you next time.

Cheers,
Valentin

P.S. Currently I am working on documenting both the new DSO GUI and
the python tooling. I hope eventually they will pass the "noob test"
:P. But, as you may have recognized above, I am not the best explainer.

***** Footnotes
[fn:4] I really appreciate c++ as a compiled language.

[fn:3] See the KStars Handbook.

[fn:2] References to objects in linked lists are stable.

[fn:1] As a matter of fact, I set out with the goal not to do any
manual memory management and not to use a single pointer in the new
code. I have been successful thus far if you would be so lenient not
to count glue code for legacy KStars systems.

** Small Insights                                                  :@Tricks:
*** How to use the Systemd userspace DBus API on Traivis-CI       :DBUS:CI:
:PROPERTIES:
:EXPORT_FILE_NAME: sysduser
:EXPORT_DATE: [2020-07-11 14:00]
:END:
I am currently working on a project which involves talking to the
~systemd~ userspace session via the session ~dbus~ instance.

After some fiddling around and enabling debug mode on travis via the
excellent user support, I came up with the following.

Travis uses VMs that run ~ubuntu~ which comes with ~systemd~.  To
enable the userspace ~dbus~ session, one has to install the
~dbus-user-session~ package. After the installation, it has to be
activated through ~systemctl --user start dbus~. Furthermore one has
to set the ~DBUS_SESSION_BUS_ADDRESS~ environment variable through
~export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus~.

TL;DR
#+begin_src yaml
script:
  - sudo apt update
  - sudo apt install dbus-user-session
  - systemctl --user start dbus
  - export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus
#+end_src

*** Fixing Linux Dualboot: Reinstalling the Windows EFI Bootloader Files
:PROPERTIES:
:EXPORT_FILE_NAME: dualboot
:EXPORT_DATE: [2020-07-11 15:00]
:END:
Note to my future self :).

Reloading my Linux install after a pretty radical 'nuke and pave' I
had to get my Windows dualboot back to work.  There are a thousand
guides on how to do that, but I'll add another one in case your setup
is similar to mine.

I have installed windows on a separate drive and Linux on my main
drive, along with the efi partition.

Don't follow this guide blindly. Think about every step you take,
because you can seriously mess up your system :).

With that out of the way, the things you have to do are:
 1. Boot a windows install medium.
 2. Choose your language and enter the 'repair options'.
 3. Go to advanced and select 'command line'.
 4. To mount the efi partition type diskpart and in diskpart then type
    list volume. A list of volumes will be printed and one of them the
    efi partition (usually around 500mb ). Select this partition
    (select volume ~[number]~) and assign a drive letter (~X~ is the
    drive letter you assign).
 5. Check where your windows partition is mounted. The diskpart list
    volume output will probably include it. I will assume that it is
    volume ~C~. Exit diskart with ~exit~.
 6. To finally install the boot files type the command ~bcdboot
    c:\windows /s x:~. This will generate boot files based on
    ~c:\windows~ and install them on the partition with the letter
    ~X~.

Thats it, you can reboot now.  You may have to reconfigure grub (or
whatever loader you use). On arch-linux, make sure you have os-prober
installed :).
*** Installing without Fear
:PROPERTIES:
:EXPORT_FILE_NAME: inst_without_fear
:EXPORT_DATE: [2020-09-16 15:00]
:END:
Note to self:

If you want to make sure some nice GNU/Linux installer does not touch
certain drives just run ~echo 1 > /sys/block/sdX/device/delete~ in a
**root** shell and the drive will vanish from the system.

Shamelessly stolen from:
https://askubuntu.com/questions/554398/how-do-i-permanently-disable-hard-drives

* Local Vars
# Local Variables:
# eval: (org-hugo-auto-export-mode)
# org-download-image-dir: "./static/images"
# org-download-heading-lvl: 3
# End: