August 05, 2020

hackergotchi for Holger Levsen

Holger Levsen

20200805-debconf7

DebConf7

This tshirt is 13 years old and from DebConf7.

DebConf7 was my 5th DebConf and took place in Edinburgh, Scotland.

And finally I could tell people I was a DD :-D Though as you can guess, that's yet another story to be told. So anyway, Edinburgh.

I don't recall exactly whether the video team had to record 6 or 7 talk rooms on 4 floors, but this was probably the most intense set up we ran. And we ran a lot, from floor to floor, and room to room.

DebConf7 was also special because it had a very special night venue, which was in an ex-church in a rather normal building, operated as sort of community center or some such, while the old church interior was still very much visible as in everything new was build around the old stuff.

And while the night venue was cool, it also ment we (video team) had no access to our machines over night (or for much of the evening), because we had to leave the university over night and the networking situation didn't allow remote access with the bandwidth needed to do anything video.

The night venue had some very simple house rules, like don't rearrange stuff, don't break stuff, don't fix stuff and just a few little more and of course we broke them in the best possible way: Toresbe with the help of people I don't remember fixed the organ, which was broken for decades. And so the house sounded in some very nice new old tune and I think everybody was happy we broke that rule.

I believe the city is really nice from the little I've seen of it. A very nice old town, a big castle on the hill :) I'm not sure whether I missed the day trip to Glasgow to fix video things or to rest or both...

Another thing I missed was getting a kilt, for which Phil Hands made a terrific design (or maybe he didn't and found someone to do it), which spelled Debian in morse code. That was pretty cool and the kilts are really nice on DebConf group pictures since then. And if you've been wearing this kilt regularily for the last 13 years it was probably also a sensible investment. ;)

It seems I don't have that many more memories of this DebConf, British power plugs and how to hack them comes to my mind and some other stuff here and there, but I remember less than previous years. I'm blaming this on the intense video setup and also on the sheer amount of people, which was the hightest until then and for some years, I believe maybe even until Heidelberg 8 years later. IIRC there were around 470 people there and over my first five years of DebConf I was incredible lucky to make many friends in Debian, so I probably just hung out and had good times.

05 August, 2020 11:20PM

20200804-debconf6

DebConf6

This tshirt is 14 years old and from DebConf6.

DebConf6 was my 4th DebConf and took place in Oaxtepec, Mexico.

I'm a bit exhausted right now which is probably quite fitting to write something about DebConf6... many things in life are a question of perception, so I will mention the waterfall and the big swirl and the band playing with the fireworks during the conference dinner, the joy that we finally could use the local fiber network (after asking for months) just after discovering that the 6h shopping tour forgot to bring the essential pig tail connectors to connect the wireless antennas to the cards, which we needed to provide network to the rooms where the talks would take place.

DebConf6 was the first DebConf with live streaming using dvswitch (written by Ben Hutchings and removed from unstable in 2015 as the world had moved to voctomix, which is yet another story to be told eventually). The first years (so DebConf6 and some) the videoteam focussed on getting the post processing done and the videos released, and streaming was optional, even though it was an exciting new feature and we still managed to stream mostly all we recorded and sometimes more... ;)

Setting up the network uplink also was very challenging and took, I don't remember exactly, until day 4 or 5 of DebCamp (which lasted 7 days), so there were group of geeks in need of network, and mostly unable to fix it, because for fixing it we needed to communicate and IRC was down. (There was no mobile phone data at that time, the first iphone wasn't sold yet, it were the dark ages.)

I remember literally standing on a roof to catch the wifi signal and excitingly shouting "I got one ping back! ... one ping back ...", less excitingly. I'll spare you the details now (and me writing them down) but I'll say that the solution involved Neil McGovern climbing an antenna and attaching a wifi antenna up high, probably 15m or 20m or some such. Finally we had uplink. I don't recall if that pig tail connector incident happened before of after, but in the end the network setup worked nicely on the wide area we occupied. Even though in some dorms the cleaning people daily removed one of our APs to be able to watch TV while cleaning ;) (Which kind of was ok, but still... they could have plugged it back in.)

I also joyfully remember a certain vegetarian table, a most memorable bus ride (I'll just say 5 or rather cinco, and, unrelated except on the same bus ride, "Jesus" (and "Maria" for sure..)!) and talking with Jim Gettys and thus learning about the One Laptop per Child (OLPC) project.

As for any DebConf, there's sooo much more to be told, but I'll end here and just thank Gunnar Wolf (as he masterminded much of this DebConf) and go to bed now :-)

05 August, 2020 11:20PM

Debian Community News

Debian, Chris Lamb, NXIVM sex cult prosecution

There have been many observations about the particularly sinister blackmailing of volunteers in Debian. It is notable that this took place at the time that a volunteer died, when another volunteer had lost a family member and by an even more bizarre coincidence, during the ongoing prosecution of leaders of the NXIVM sex cult in New York.

As racy details of the NXIVM abuses trickled out of the courtroom and into the newspapers each day, volunteers noted the similarities to how the Debian organization had been acting under the leadership of Chris Lamb.

A power couple

NXIVM was operated by Keith Raniere in collusion with his associate/girlfriend, the actress Allison Mack. Debian was officially led by Chris Lamb, while his girlfriend Molly de Blanc had created the infamous Anti-Harassment team, analogous to Scientology's Sea Organization, to promote submissiveness and obedience under the guise of a "Code of Conduct".

Branding people

In NXIVM, Raniere and Mack were branding their victims. They burnt their initials, KR & AM, in close proximity to victims' genitalia.

Early in 2018, Alexander "formorer" Wirth had set up Debian's public Git repositories, hosted in the Salsa.debian.org service. Shortly after this, at DebConf18 in Taiwan, Lamb had started the discussions about how to brand volunteers with adverse records in Git / Salsa. A few days before Christmas, this weapon was unleashed on Dr Norbert Preining, who maintains the LaTeX packages used widely in the academic world.

debian shaming branding norbert preining chris lamb molly de blanc mollamby debian shaming branding norbert preining chris lamb molly de blanc mollamby

These intentionally permanent scars in Git, mailing list archives and the Debian Bug Tracking System are functionally equivalent to the branding of NXIVM victims.

When volunteers have filed GDPR requests for the deletion of these records, Debian oligarchs have stonewalled and used Debian money to hire lawyers to perpetuate the abuse.

Joerg Jaspert: This is not involving anything from the universal declaration of human rights.
Miriam Ballhausen Bird & Bird twobirds gdpr lawyer abuse harassment of debian volunteer

Destroying people

Just as Debian oligarchs often use nicknames and acronyms, NXIVM's founder, Raniere, had chosen to hide behind the pseudonym Vanguard, taken from an arcade game in which the destruction of one's enemies increased one's own power. It is a remarkable parallel to the style used by some of the worst leaders in Debian over the years.

In fact, every year there are public discussions about who to kick out of Debian. Enrico Zini, one of the Debian Account Managers who is currently engaged in blackmailing a volunteer, asked candidates in the 2006 leadership election to publicly name five people they would expel.

enrico zini harassment abuse volunteers

The people who write things like this have never done any real work themselves. If they had, they might understand that this is not the right way to thank volunteers for years of contributions to Debian.

molly de blanc cyberbully harassing volunteers

Can you imagine any other organization where participants egg each other on to publicly denounce volunteers, or is this only possible in the world of Scientology, NXIVM and Debian?

debian nxivm scientology

Keeping dossiers on people

Raniere & Co had a practice of building dossiers on people. Lamb's girlfriend, de Blanc, boasted about the same practice, whispering networks, in her FOSDEM 2019 talk about being an enforcer. The debian-private leaks show that this has been going on in Debian for decades.

In an email leaked from FSFE, another organization racked by scandals, the FSFE president Matthias Kirschner relates a conversation he had with Chris Lamb:

One general wish -- which I agreed with -- from Debian was to better share information about people

The first conviction in the NXIVM case was that of Nancy Salzman, who pleaded guilty to racketeering conspiracy, tearfully confessed in court on March 13 that she tracked and monitored the usernames and passwords of suspected moles in the group to ensure they weren’t leaking details about the group’s inner workings.

Secret shame

NXIVM pressured their victims to provide nude photos, which were held as a form of collatoral.

Lamb, de Blanc and their associates, the Anti-Harassment team and Debian Account Managers, had been making secret findings of misconduct against volunteers and then making veiled threats to disclose these abusive decisions if the volunteers were not submissive enough. Dr Preining called their bluff by releasing a bundle of their nasty emails himself.

Slavery

Slavery and modern slavery mean different things in different contexts. Salzman's daughter admitted in court that she kept another woman as a slave.

Some of Debian's Google Summer of Code interns have come forward with allegations of non-payment. In 2018, one intern filed a complaint with Debian and then escalated to Stephanie Taylor, Google's head of the GSoC program, after his mentor pushed him to work up to the last day of the internship and then withheld payment. Taylor refused to assist the intern. (Note: the student chose to disclose this matter publicly, we are not doing that without consent)

The two mentors for this student had disagreed: only one of them wanted to fail the intern. The other one felt the case justified closer consideration. Molly de Blanc overrode him as she didn't want to bother Google with news of the childish bickering that plagues Debian. de Blanc then received a free trip to the GSoC mentor summit in California, funded by Google.

Google and Taylor are at the root of the slavery problem, insisting that the interns be referred to as students as a dishonest fudge to deny their status as workers.

The intern in question is from Bhopal, one of the most impoverished areas of India, decimated by the 1984 industrial disaster that has become synonymous with the name of the city.

Blackmail

The charges that eventually led to NXIVM jail time were on the basis of sex trafficking. We are not talking about teenage girls bought from Thailand. The victims of this sex trafficking program were educated and wealthy adult women, coerced into submission by blackmail.

The Code of Conduct enforcement mantras pushed by Molly de Blanc, in comparison, are aimed squarely at educated and wealthy adult men. Once again, the force of blackmail and fear of humiliation is used to deny people their freedom, as in the case of Dr Preining and other victims.

We can prove that this is blackmail very clearly: the message disclosed by Dr Norbert Preining shows that Debian oligarchs still expected him to maintain his packages after his secret expulsion. This factor makes the abuse indistinguishable from the blackmailing of NXIVM victims with nude photos.

Both NXIVM victims and Debian victims reported being coerced to do tasks for their would-be masters in much the same way.

NXIVM's slaves did dishwashing and similar chores. Dr Preining's work on the TeX-live packages is widely used in the international community of academics and researchers. Many now feel uncomfortable about how this software is produced but have little option to work around it.

Debian, under Chris Lamb and Molly de Blanc, had become nothing better than a revenge porn syndicate.

Enslaving women

Many of NXIVMs victims are women. Debian has been unable to attract women to enter as volunteers so they began enticing them into Debian with Outreachy internships. Debian's community of volunteers has never had more than two percent female participation. While GSoC money is from Google's bank account, the Outreachy stipends are paid from Debian's own bank accounts, held by SPI Inc. Debian's constitution states that contributors must be volunteers but the oligarchs have started using funds to pay women to volunteer. Multiple women have reported being threatened or coerced to behave in a particular way or they won't continue to receive travel funding for Debian's events. Some had even heard the case of the intern from Bhopal who never got paid.

In one disturbing case, multiple people have alleged somebody from Canonical Ltd, the company behind Ubuntu, was able to enter Debian as an official GSoC mentor and pursue a liaison with one of the women from a low income country attending DebConf on a diversity bursary.

NXIVM members spoke about building a dungeon where women could volunteer to be abused.

The belief that women enter such relationships and dungeons voluntarily and of their own free will is a dubious fantasy.

One of Debian's GSoC admins resigned in August 2018 almost immediately after hearing about the relationship. Rather than listening to his concerns, oligarchs started spreading rumours to try and undermine him.

Blaming and shaming victims

Due to the sexual nature of NXIVM crimes, the court suppressed the names of the victims from identification in the press. Raniere fought tooth and nail to try and have them named, hoping to cause embarassment.

Both previous Debian Project Leaders, Chris Lamb and Sam Hartman have followed in Raniere's footsteps by making plots to shame and humiliate volunteers who speak up about cult phenomena in Debian.

Rituals where victims submit to the oligarchs

Lauren Salzman, whose mother co-founded NXIVM, recalls reciting the following self-deprecating oath when being branded:

Master, please brand me, It would be an honor. An honor that I want to wear for the rest of my life.

Dr Norbert Preining posted similar words in the very public debian-project mailing list after three months of blackmail that occurred concurrently with the NXIVM trial:

I also will take care to listen carefully to advice and corrections, ...
In cooperation with DAM - and the invaluable help of some fellow DDs - we have reached the agreement about my further status. DAM will write about this in a separate email outlining the agreement and consequences.

A human rights perspective

From the Universal Declaration of Human Rights:

Article 12: No one shall be subjected to arbitrary interference with his privacy, family, home or correspondence, nor to attacks upon his honour and reputation. Everyone has the right to the protection of the law against such interference or attacks.

and the European Convention on Human Rights:

Article 8 – Right to respect for private and family life. Everyone has the right to respect for his private and family life, his home and his correspondence.

It is clear that the communications circulated and perpetuated by Mollamby and their associates, backed by the weight of Debian's reputation, are a violation of these rights.

The UN's Special Rapporteur on Torture and other Cruel, Inhumane or Degrading Treatment or Punishment, Professor Nils Melzer, was no doubt thinking about organizations like Google and the way they use fronts like Debian and FSFE to shame their critics when he wrote that these organizations:

not only have the capacity to conduct cyber-operations inflicting severe suffering on countless individuals, but may well decide to do so for any of the purposes of torture. Cybertechnology can also be used to inflict, or contribute to, severe mental suffering while avoiding the conduit of the physical body, most notably through intimidation, harassment, surveillance, public shaming and defamation, as well as appropriation, deletion or manipulation of information.

The Debian mailing lists have now been moderated/censored to avoid questions about these matters. Please remember to follow the Uncensored Debian Planet site (use the RSS feed) to ensure you have access to facts like these from all sides.

Would you like to help? Please share the opportunity for an intern to study cult phenomena in Debian.

05 August, 2020 02:05PM

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

RcppCCTZ 0.2.8: Minor API Extension

A new minor release 0.2.8 of RcppCCTZ is now on CRAN.

RcppCCTZ uses Rcpp to bring CCTZ to R. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. And while CCTZ is made by Google(rs), it is not an official Google product. The RcppCCTZ page has a few usage examples and details. This package was the first CRAN package to use CCTZ; by now at least three others do—using copies in their packages which remains less than ideal.

This version adds three no throw variants of three existing functions, contributed again by Leonardo. This will be used in an upcoming nanotime release which we are finalising now.

Changes in version 0.2.8 (2020-08-04)

  • Added three new nothrow variants (for win32) needed by the expanded nanotime package (Leonardo in #37)

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

05 August, 2020 01:25AM

hackergotchi for Holger Levsen

Holger Levsen

20200801-debconf3

DebConf3

This tshirt is 17 years old and from DebConf3. I should probably wash it at 60 celcius for once...

DebConf3 was my first DebConf and took place in Oslo, Norway, in 2003. I was very happy to be invited, like any Debian contributor at that time, and that Debian would provide food and accomodation for everyone. Accomodation was sleeping on the floor in some classrooms of an empty school and I remember having tasted grasshoppers provided by a friendly Gunnar Wolf there, standing in line on the first day with the SSH maintainer (OMG!1 (update: I originally wrote here that it wasn't Colin back then, but Colin mailed me to say that he was indeed maintaining SSH even back then, so I've met a previous maintainer there)) and meeting the one Debian person I had actually worked with before: Thomas Lange or MrFAI (update: Thomas also mailed me and said this was at DebConf5). In Oslo I also was exposed to Skolelinux / Debian Edu for the first time, saw a certain presentation from the FTP masters and also noticed some people recording the talks, though as I learned later these videos were never released to the public. And there was this fiveteen year old called Toresbe, who powered on the PDP's which were double his age. And then actually made use of them. And and and.

I'm very happy I went to this DebConf. Without going my Debian journey would have been very different today. Thanks to everyone who made this such a welcoming event. Thanks to anyone who makes any event welcoming! :)

05 August, 2020 12:28AM

August 04, 2020

Osamu Aoki

exim4 configuration for Desktop (better gmail support)

Since gmail rewrites "From:" address now (2020) and keep changing access limitation, it is wise not  to use it as smarthost any more.  (If you need to access multiple gmail addresses from mutt etc, use esmtp etc.)

---
For most of our Desktop PC running with stock exim4 and mutt, I think sending out mail is becoming a bit rough since using random smarthost causes lots of trouble due to the measures taken to prevent spams.

As mentioned in Exim4 user FAQ , /etc/hosts should have FQDN with external DNS resolvable domain name listed instead of localdomain to get the correct EHLO/HELO line.  That's the first step.

The stock configuration of exim4 only allows you to use single smarthost for all your mails.  I use one address for my personal use which is checked by my smartphone too.  The other account is for subscribing to the mailing list.  So I needed to tweak ...

Usually, mutt is smart enough to set the From address since my .muttrc has

# Set default for From: for replyes for alternates.
set reverse_name

So how can I teach exim4 to send mails depending on the  mail accounts listed in the From header.

For my gmail accounts, each mail should be sent to the account specific SMTP connection matching your From header to get all the modern SPAM protection data in right state.  DKIM, SPF, DMARC...  (Besides, they overwrite From: header anyway if you use wrong connection.)

For my debian.org mails, mails should be sent from my shell account on people.debian.org so it is very unlikely to be blocked.  Sometimes, I wasn't sure some of these debian.org mails sent through my ISP's smarthost are really getting to the intended person.

To these ends, I have created small patches to the /etc/exim4/conf.d files and reported it to Debian BTS: #869480 Support multiple smarthosts (gmail support).  These patches are for the source package.

To use my configuration tweak idea, you have easier route no matter which exim version you are using.  Please copy and read pertinent edited files from my github site to your installed /etc/exim4/conf.d files and get the benefits.
If you really wish to keep envelope address etc. to match From: header, please rewite agressively using the From: header using eddited rewrite/31_exim4-config_rewriting as follows:

.ifndef NO_EAA_REWRITE_REWRITE
*@+local_domains "${lookup{${local_part}}lsearch{/etc/email-addresses}\
                   {$value}fail}" f
# identical rewriting rule for /etc/mailname
*@ETC_MAILNAME "${lookup{${local_part}}lsearch{/etc/email-addresses}\
                   {$value}fail}" f
.endif
* "$h_from:" Frs

So far its working fine for me but if you find bug, let me know.

Osamu

04 August, 2020 03:03PM by osamu.aoki@gmail.com (noreply@blogger.com)

August 03, 2020

hackergotchi for Holger Levsen

Holger Levsen

20200803-debconf5

DebConf5

This tshirt is 15 years old and from DebConf5. It still looks quite nice! :)

DebConf5 was my 3rd DebConf and took place in Helsinki, or rather Espoo, in Finland.

This was one of my most favorite DebConfs (though I basically loved them all) and I'm not really sure why, I guess it's because of the kind of community at the event. We stayed in some future dorms of the universtity, which were to be first used by some European athletics chamopionship and which we could use even before that, guests zero. Being in Finland there were of course saunas in the dorms, which we frequently used and greatly enjoyed. Still, one day we had to go on a trip to another sauna in the forest, because of course you cannot visit Finland and only see one sauna. Or at least, you should not.

Another aspect which increased community bonding was that we had to authenticate using 802.10 (IIRC, please correct me) which was an authentication standard mostly used for wireless but which also works for wired ethernet, except that not many had used it on Linux before. Thus quite some related bugs were fixed in the first days of DebCamp...

Then my powerpc ibook also decided to go bad, so I had to remove 30 screws to get the harddrive out and 30 screws back in, to not have 30 screws laying around for a week. Then I put the harddrive into a spare (x86) laptop and only used my /home partition and was very happy this worked nicely. And then, for travelling back, I had to unscrew and screw 30 times again. (I think my first attempt took 1.5h and the fourth only 45min or so ;) Back home then I bought a laptop where one could remove the harddrive using one screw.

Oh, and then I was foolish during the DebConf5 preparations and said, that I could imagine setting up a team and doing video recordings, as previous DebConfs mostly didn't have recordings and the one that had, didn't have releases of them...

And so we did videos. And as we were mostly inexperienced we did them the hard way: during the day we recorded on tape and then when the talks were done, we used a postprocessing tool called 'cinelerra' and edited them. And because Eric Evans was on the team and because Eric worked every night almost all night, all nights, we managed to actually release them all when DebConf5 was over. I very well remember many many (23 or 42) Debian people cleaning the dorms thoroughly (as they were brand new..) and Eric just sitting somewhere, exhausted and watching the cleaners. And everybody was happy Eric was idling there, cause we knew why. In the aftermath of DebConf5 Ben Hutchings then wrote videolink (removed from sid in 2013) which we used to create video DVDs of our recordings based on a simple html file with links to the actual videos.

There were many more memorable events. The boat ride was great. A pirate flag appeared. One night people played guitar until very late (or rather early) close to the dorms, so at about 3 AM someone complained about it, not in person, but on the debian-devel mailinglist. And those drunk people playing guitar, replied immediatly on the mailinglist. And then someone from the guitar group gave a talk, at 9 AM, and the video is online... ;) (It's a very slowwwwwww talk.)

If you haven't been to or close to the polar circles it's almost impossible to anticipate how life is in summer there. It get's a bit darker after midnight or rather after 1 AM and then at 3 AM it get's light again, so it's reaaaaaaally easy to miss the night once and it's absolutly not hard to miss the night for several nights in a row. And then I shared a room with 3 people who all snore quite loud...

There was more. I was lucky to witness the first (or second?) cheese and whine party which at that time took place in a dorm room with, dunno 10 people and maybe 15 kinds of cheese. And, of course, I met many wonderful people there, to mention a few I'll say Jesus, I mean mooch or data, Amaya and p2. And thanks to some bad luck which turned well, I also had my first time ever Sushi in Helsinki.

And and and. DebConfs are soooooooo good! :-) I'll stop here as I originally planned to only write a paragraph or two about each and there are quite some to be written!

Oh, and as we all learned, there are probably no mosquitos in Helsinki, just in Espoo. And you can swim naked through a lake and catch a taxi on the other site, with no clothes and no money, no big deal. (And you might not believe it, but that wasn't me. I cannot swim that well.)

03 August, 2020 10:16PM

Sylvain Beucler

Debian LTS and ELTS - July 2020

Debian LTS Logo

Here is my transparent report for my work on the Debian Long Term Support (LTS) and Debian Extended Long Term Support (ELTS), which extend the security support for past Debian releases, as a paid contributor.

In July, the monthly sponsored hours were split evenly among contributors depending on their max availability - I was assigned 25.25h for LTS (out of 30 max; all done) and 13.25h for ELTS (out of 20 max; all done).

We shifted suites: welcome Stretch LTS and Jessie ELTS. The LTS->ELTS switch happened at the start of the month, but the oldstable->LTS switch happened later (after finalizing and flushing proposed-updates to a last point release), causing some confusion but nothing major.

ELTS - Jessie

  • New local build setup
  • ELTS buildds: request timezone harmonization
  • Reclassify in-progress updates from jessie-LTS to jessie-ELTS
  • python3.4: finish preparing update, security upload ELA 239-1
  • net-snmp: global triage: bisect CVE-2019-20892 to identify affected version, jessie/stretch not-affected
  • nginx: global triage: clarify CVE-2013-0337 status; locate CVE-2020-11724 original patch and regression tests, update MITRE
  • nginx: security upload ELA-247-1 with 2 CVEs

LTS - Stretch

  • Reclassify in-progress/needed updates from stretch/oldstable to stretch-LTS
  • rails: upstream security: follow-up on CVE-2020-8163 (RCE) on upstream bug tracker and create pull request for 4.x (merged), hence getting some upstream review
  • rails: global security: continue coordinating upload in multiple Debian versions, prepare fixes for common stretch/buster vulnerabilities in buster
  • rails: security upload DLA-2282 fixing 3 CVEs
  • python3.5: security upload DLA-2280-1 fixing 13 pending non-critical vulnerabilities, and its test suite
  • nginx: security upload DLA-2283 (cf. common ELTS work)
  • net-snmp: global triage (cf. common ELTS work)
  • public IRC monthly team meeting
  • reach out to clarify the intro from last month's report, following unsettled feedback during meeting

Documentation/Scripts

  • ELTS/README.how-to-release-an-update: fix typo
  • ELTS buildd: attempt to diagnose slow perfs, provide comparison with Debian and local builds
  • LTS/Meetings: improve presentation
  • SourceOnlyUpload: clarify/de-dup pbuilder doc
  • LTS/Development: reference build logs URL, reference proposed-updates issue during dists switch, reference new-upstream-versioning discussion, multiple jessie->stretch fixes and clean-ups
  • LTS/Development/Asan: drop wheezy documentation
  • Warn about jruby mis-triage
  • Provide feedback for ksh/CVE-2019-14868
  • Provide feedback for condor update
  • LTS/TestsSuites/nginx: test with new request smuggling test cases

03 August, 2020 01:52PM

August 02, 2020

Enrico Zini

hackergotchi for Sean Whitton

Sean Whitton

GNU Emacs' Transient Mark mode

Something I’ve found myself doing as the pandemic rolls on is picking out and (re-)reading through sections of the GNU Emacs manual and the GNU Emacs Lisp reference manual. This has got me (too) interested in some of the recent history of Emacs development, and I did some digging into archives of emacs-devel from 2008 (15M mbox) regarding the change to turn Transient Mark mode on by default and set mark-even-if-inactive to true by default in Emacs 23.1.

It’s not always clear which objections to turning on Transient Mark mode by default take into account the mark-even-if-inactive change. I think that turning on Transient Mark mode along with mark-even-if-inactive is a good default. The question that remains is whether the disadvantages of Transient Mark mode are significant enough that experienced Emacs users should consider altering Emacs’ default behaviour to mitigate them. Here’s one popular blog arguing for some mitigations.

How might Transient Mark mode be disadvantageous?

The suggestion is that it makes using the mark for navigation rather than for acting on regions less convenient:

  1. setting a mark just so you can jump back to it (i) is a distinct operation you have to think of separately; and (ii) requires two keypresses, C-SPC C-SPC, rather than just one keypress

  2. using exchange-point-and-mark activates the region, so to use it for navigation you need to use either C-u C-x C-x or C-x C-x C-g, neither of which are convenient to type, or else it will be difficult to set regions at the place you’ve just jumped to because you’ll already have one active.

There are two other disadvantages that people bring up which I am disregarding. The first is that it makes it harder for new users to learn useful ways in which to use the mark when it’s deactivated. This happened to me, but it can be mitigated without making any behavioural changes to Emacs. The second is that the visual highlighting of the region can be distracting. So far as I can tell, this is only a problem with exchange-point-and-mark, and it’s subsumed by the problem of that command actually activating the region. The rest of the time Emacs’ automatic deactivation of the region seems sufficient.

How might disabling Transient Mark mode be disadvantageous?

When Transient Mark mode is on, many commands will do something usefully different when the mark is active. The number of commands in Emacs which work this way is only going to increase now that Transient Mark mode is the default.

If you disable Transient Mark mode, then to use those features you need to temporarily activate Transient Mark mode. This can be fiddly and/or require a lot of keypresses, depending on exactly where you want to put the region.

Without being able to see the region, it might be harder to know where it is. Indeed, this is one of the main reasons for wanting Transient Mark mode to be the default, to avoid confusing new users. I don’t think this is likely to affect experienced Emacs users often, however, and on occasions when more precision is really needed, C-u C-x C-x will make the region visible. So I’m not counting this as a disadvantage.

How might we mitigate these two sets of disadvantages?

Here are the two middle grounds I’m considering.

Mitigation #1: Transient Mark mode, but hack C-x C-x behaviour

(defun spw/exchange-point-and-mark (arg)
  "Exchange point and mark, but reactivate mark a bit less often.

Specifically, invert the meaning of ARG in the case where
Transient Mark mode is on but the region is inactive."
  (interactive "P")
  (exchange-point-and-mark
   (if (and transient-mark-mode (not mark-active))
       (not arg)
     arg)))
(global-set-key [remap exchange-point-and-mark] 'spw/exchange-point-and-mark)

We avoid turning Transient Mark mode off, but mitigate the second of the two disadvantages given above.

I can’t figure out why it was thought to be a good idea to make C-x C-x reactivate the mark and require C-u C-x C-x to use the action of exchanging point and mark as a means of navigation. There needs to a binding to reactivate the mark, but in roughly ten years of having Transient Mark mode turned on, I’ve found that the need to reactivate the mark doesn’t come up often, so the shorter and longer bindings seem the wrong way around. Not sure what I’m missing here.

Mitigation #2: disable Transient Mark mode, but enable it temporarily more often

(setq transient-mark-mode nil)
(defun spw/remap-mark-command (command &optional map)
  "Remap a mark-* command to temporarily activate Transient Mark mode."
  (let* ((cmd (symbol-name command))
         (fun (intern (concat "spw/" cmd)))
         (doc (concat "Call `"
                      cmd
                      "' and temporarily activate Transient Mark mode.")))
    (fset fun `(lambda ()
                 ,doc
                 (interactive)
                 (call-interactively #',command)
                 (activate-mark)))
    (if map
        (define-key map (vector 'remap command) fun)
      (global-set-key (vector 'remap command) fun))))

(dolist (command '(mark-word
                   mark-sexp
                   mark-paragraph
                   mark-defun
                   mark-page
                   mark-whole-buffer
                   rectangle-mark-mode))
  (spw/remap-mark-command command))
(with-eval-after-load 'org
  (spw/remap-mark-command 'org-mark-element org-mode-map)
  (spw/remap-mark-command 'org-mark-subtree org-mode-map))

;; sometimes a key to just activate the mark is wanted
(global-set-key "\M-i" (lambda () (interactive) (activate-mark)))
;; resettle the previous occupant
(global-set-key "\M-I" #'tab-to-tab-stop)

Here we remove both of the disadvantages of Transient Mark mode given above, and mitigate the main disadvantage of not activating Transient Mark mode by making it more convenient to activate it temporarily.

For example, this enables using C-M-SPC C-M-SPC M-( to wrap the following two function arguments in parentheses. And you can hit M-h a few times to mark some blocks of text or code, then operate on them with commands like M-% and C-/ which behave differently when the region is active.1

Comparing these mitigations

Both of these mitigations handle the second of the two disadvantages of Transient Mark mode given above. What remains, then, is

  1. under the effects of mitigation #1, how much of a barrier to using marks for navigational purposes is it to have to press C-SPC C-SPC instead of having a single binding, C-SPC, for all manual mark setting2

  2. under the effects of mitigation #2, how much of a barrier to taking advantage of commands which act differently when the region is active is it to have to temporarily enable Transient Mark mode with C-SPC C-SPC, M-i or one of the mark-* commands?

These are unknowns.3 So I’m going to have to experiment, I think, to determine which mitigation to use, if either. In particular, I don’t know whether it’s really significant that setting a mark for navigational purposes and for region marking purposes are distinct operations under mitigation #1.

My plan is to start with mitigation #2 because that has the additional advantage of allowing me to confirm or disconfirm my belief that not being able to see where the region is will only rarely get in my way.

Update 23/Jul/2020: A little less than two months later, mitigation #2 has worked out so well that I do not intend to try out mitigation #1. Transient Mark mode gets activated pretty much whenever it needs to be, mainly by means of M-h and C-M-SPC, and occasionally by C-SPC C-SPC or M-i, without much effort. I’ve been using marks for navigation effectively, except for struggling to set quite enough marks – I keep thinking “time to jump back … oh, once again, I failed to set a mark before coming here.” This seems a surmountable difficulty.


  1. The idea of making the mark-* commands activate the mark comes from an emacs-devel post by Stefan Monnier in the archives linked above.
  2. One remaining possibility I’m not considering is mitigation #1 plus binding something else to do the same as C-SPC C-SPC. I don’t believe there are any easily rebindable keys which are easier to type than typing C-SPC twice. And this does not deal with the two distinct mark-setting operations problem.
  3. Another way to look at this is the question of which of setting a mark for navigational purposes and activating a mark should get C-SPC and which should get C-SPC C-SPC.

02 August, 2020 03:31PM

Enrico Zini

Libreoffice presentation tips

Snap guides

Dragging from the rulers does not always create snap guides. If it doesn't, click on the slide background, "Snap guides", "Insert snap guide". In my case, after the first snap guide was manually inserted, it was possible to drag new one from the rulers.

Master slides

How to edit a master slide

  • Show master slides side pane
  • Right click on master slide
  • Edit Master...
  • An icon appears in the toolbar: "Close Master View"
  • Apply to all slides might not apply to the first slide created as the document was opened

Change styles in master slide

Do not change properties of text by selecting placeholder text in the Master View. Instead, open the Styles and formatting sidebar, and edit the styles in there.

This means the style changes are applied to pages in all layouts, not just the "Title, Content" layout that is the only one editable in the "Master View".

How to duplicate a master slide

There seems to be no feature implemented for this, but you can do it, if you insist:

  • Save a copy of the document
  • Rename the master slide
  • Drag a slide, that uses the renamed master slide, from the copy of the document to the original one

It's needed enough that someone made a wikihow: https://www.wikihow.com/Copy-a-LibreOffice-Impress-Master-Slide archive.org

How to change the master slide for a layout that is not "Title, Content"

I could not find a way to do it, but read on for a workaround.

I found an ask.libreoffice.org question that went unanswered.

I asked on #libreoffice on IRC and got no answer:

Hello. I'm doing the layout for a presentation in impress, and I can edit all sorts of aspects of the master slide. It seems that I can only edit the "Title, Content" layout of the master slide, though. I'd like to edit, for example, the "Title only" layout so that the title appears in a different place than the top of the page. Is it possible to edit specific layouts in a master page?

In the master slide editor it seems impossible to select a layout, for example.

Alternatively I tried creating multiple master slides, but then if I want to create a master slide for a title page, there's no way to remove the outline box, or the title box.

My work around has been to create multiple master slides, one for each layout. For a title layout, I moved the outline box into a corner, and one has to remove it manually after create a new slide.

There seems to be no way of changing the position of elements not found in the "Title, Content" layout, like "Subtitle". On the other hand, given that one's working with an entirely different master slide, one can abuse the outline box as a subtitle.

Note that if you later decide to change a style element for all the slides, you'll need to go propagate the change to the "Styles and Formatting" menu of all master slides you're using.

02 August, 2020 01:00PM

Andrew Cater

Debian 10.5 media testing - 202001082250 - last few debian-live images being tested for amd64 - Calamares issue - Post 5 of several.

Last few debian-live images being tested for amd64. We have found a bug with the debian-live Gnome flavour. This specifically affects installs after booting from the live media and then installing to the machine using  the Calamares installer found on the desktop. The bug was introduced as a fix for one issue that has produced further buggy behaviour as a result.

Fixes are known - we've had highvoltage come and debug them with us - but will not be put out with this release but will wait for the 10.6 release which will allow for a longer time for debugging overall.

You can still run from the live-media, you can still install with the standard Debian installers found in the menu of the live-media disk - this is _only_ a limited time issue with the Calamares installer. At this point in the release cycle, it's been judged better to release the images as they are - with known and documented issues - than to try and debug them in a hurry and risk damaging or delaying a stable point release.

02 August, 2020 12:59PM by Andrew Cater (noreply@blogger.com)

Enrico Zini

Gender, inclusive communities, and dragonflies

From https://en.wikipedia.org/wiki/Dragonfly#Sex_ratios:

Sex ratios

The sex ratio of male to female dragonflies varies both temporally and spatially. Adult dragonflies have a high male-biased ratio at breeding habitats. The male-bias ratio has contributed partially to the females using different habitats to avoid male harassment.

As seen in Hine's emerald dragonfly (Somatochlora hineana), male populations use wetland habitats, while females use dry meadows and marginal breeding habitats, only migrating to the wetlands to lay their eggs or to find mating partners.

Unwanted mating is energetically costly for females because it affects the amount of time that they are able to spend foraging.

02 August, 2020 09:32AM

August 01, 2020

Molly de Blanc

busy busy

I’ve been working with Karen Sandler over the past few months on the first draft of the Declaration of Digital Autonomy. Feedback welcome, please be constructive. It’s a pretty big deal for me, and feels like the culmination of a lifetime of experiences and the start of something new.

We talked about it at GUADEC and HOPE. We don’t have any other talks scheduled yet, but are available for events, meetups, dinner parties, and b’nai mitzvahs.

01 August, 2020 09:15PM by mollydb

Andrew Cater

Debian 10.5 media testing - 202008012055 - post 4 of several

We've more or less finished testing on the Debian install images. Now moving on to the debian-live images. Bugs found and being triaged live as I type. Lots of typing and noises in the background of the video conference. Now at about 12-14 hours in on this for some of the participants. Lots of good work still going on, as ever.

01 August, 2020 09:01PM by Andrew Cater (noreply@blogger.com)

Debian 10.5 media testing - pause for supper - 202001081715 - post 3 of several

Various of the folk doing this have taken a food break until 1900 local. A few glitches, a few that needed to be tried over again - but it's all going fairly well.

It is likely that at least one of the CD images will be dropped. The XFCE desktop install CD for i386 is now too large to fit on CD media. The netinst .iso files / the DVD 1 file / any of the larger files available via Jigdo will all help you achieve the same result.

There are relatively few machines that are i386 architecture only - it might be appropriate for people to use 64 bit amd64 from this point onwards as pure i386 machines are now approaching ten years old as a minimum. If you do need a graphical user environment for a pure i386 machine, it can be installed by using an expert install or using tasksel in the installation process.

01 August, 2020 05:37PM by Andrew Cater (noreply@blogger.com)

Debian 10.5 media testing - continuing quite happily - 202001081320 - post 2 of several

We've now settled into a reasonable rhythm: RattusRattus and Isy and Sledge all working away hard in Cambridge: Schweer in Germany and me here in Cheltenham.

Lots of chat backwards and forwards and a good deal of work being done, as ever.

It's really good to be back in the swing of it and we owe thanks to folk for setting up infrastructure for us to use for video chat, which makes a huge difference: even though I know what they're like, it's still good to see my colleagues.

01 August, 2020 05:24PM by Andrew Cater (noreply@blogger.com)

Debian 10.5 media testing process started 202008011145 - post 1 of several.

The media testing process has started slightly late. There will be a _long_ testing process over much of the day: the final media image releases are likely to be at about 0200-0300UTC tomorrow.

Just settling in for a long day of testing: as ever, it's good to be chatting with my Debian colleagues in Cambridge and with Schweer in Germany. It's going to be a hot one - 30 Celsius (at least) and high humidity for all of us.

EDIT: Corrected for UTC :)

01 August, 2020 01:01PM by Andrew Cater (noreply@blogger.com)

Paul Wise

FLOSS Activities July 2020

Focus

This month I didn't have any particular focus. I just worked on issues in my info bubble.

Changes

Issues

Review

Administration

  • Debian wiki: unblock IP addresses, approve accounts, reset email addresses

Communication

Sponsors

The purple-discord, ifenslave and psqlodbc work was sponsored by my employer. All other work was done on a volunteer basis.

01 August, 2020 01:02AM

hackergotchi for Junichi Uekawa

Junichi Uekawa

August and feels like it finally.

August and feels like it finally. July didn't feel like July and felt like June because it rained so much. This is summer.

01 August, 2020 12:54AM by Junichi Uekawa

July 31, 2020

hackergotchi for Ben Hutchings

Ben Hutchings

Debian LTS work, July 2020

I was assigned 20 hours of work by Freexian's Debian LTS initiative, but only worked 5 hours this month and returned the remainder to the pool.

Now that Debian 9 'stretch' has entered LTS, the stretch-backports suite will be closed and no longer updated. However, some stretch users rely on the newer kernel version provided there. I prepared to add Linux 4.19 to the stretch-security suite, alongside the standard package of Linux 4.9. I also prepared to update the firmware-nonfree package so that firmware needed by drivers in Linux 4.19 will also be available in stretch's non-free section. Both these updates will be based on the packages in stretch-backports, but needed some changes to avoid conflicts or regressions for users that continue using Linux 4.9 or older non-Debian kernel versions. I will upload these after the Debian 10 'buster' point release.

31 July, 2020 10:40PM

hackergotchi for Chris Lamb

Chris Lamb

Free software activities in July 2020

Here is my monthly update covering what I have been doing in the free and open source software world during July 2020 (previous month):

  • Opened a pull request to make the build reproducible in PyERFA, a set of Python bindings for various astronomy-related utilities (#45), as well as one for PeachPy assembler to make the output of codecode/x86_64.py reproducible (#108).
  • As part of being on the board of directors of the Open Source Initiative and Software in the Public Interest I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet, as well as the usual internal discussions regarding logistics and policy etc. This month, it was SPI's Annual General Meeting and the OSI has been running a number of remote strategy sessions for the board.

  • Fixed an issue in my tickle-me-email library that implements Getting Things Done (GTD)-like behaviours in IMAP inboxes to ensure that all messages have a unique Message-Id header. [...]

  • Reviewed and merged even more changes by Pavel Dolecek into my Strava Enhancement Suite, a Chrome extension to improve the user experience on the Strava athletic tracker.

  • Updated travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub, to use the Travis CI continuous integration platform) to fix a compatibility issue with the latest version of mk-build-deps. [...][...]

For Lintian, the static analysis tool for Debian packages:

  • Update the regular expression to search for all the released versions in a .changes file. [...]

  • Avoid false-positives when matching sensible-utils utilities such as i3-sensible-pager. (#966022)

  • Rename the send-patch tag to patch-not-forwarded-upstream. [...]

  • Drop reminders from 26 tags that false-positives should be reported to Lintian as this is implicit in all our tags. [...]


§


Reproducible Builds

One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes.

The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

The project is proud to be a member project of the Software Freedom Conservancy. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.

This month, I:


§


diffoscope

Elsewhere in our tooling, I made the following changes to diffoscope, including preparing and uploading versions 150, 151, 152, 153 & 154 to Debian:

  • New features:

    • Add support for flash-optimised F2FS filesystems. (#207)
    • Don't require zipnote(1) to determine differences in a .zip file as we can use libarchive. [...]
    • Allow --profile as a synonym for --profile=-. [...]
    • Increase the minimum length of the output of strings(1) to eight characters to avoid unnecessary diff noise. [...]
    • Drop some legacy argument styles: --exclude-directory-metadata and --no-exclude-directory-metadata have been replaced with --exclude-directory-metadata={yes,no}. [...]
  • Bug fixes:

    • Pass the absolute path when extracting members from SquashFS images as we run the command with working directory in a temporary directory. (#189)
    • Correct adding a comment when we cannot extract a filesystem due to missing libguestfs module. [...]
    • Don't crash when listing entries in archives if they don't have a listed size such as hardlinks in ISO images. (#188)
  • Output improvements:

    • Strip off the file offset prefix from xxd(1) and show bytes in groups of 4. [...]
    • Don't emit javap not found in path if it is available in the path but it did not result in an actual difference. [...]
    • Fix ... not available in path messages when looking for Java decompilers that used the Python class name instead of the command. [...]
  • Logging improvements:

    • Add a bit more debugging info when launching libguestfs. [...]
    • Reduce the --debug log noise by truncating the has_some_content messages. [...]
    • Fix the compare_files log message when the file does not have a literal name. [...]
  • Codebase improvements:

    • Rewrite and rename exit_if_paths_do_not_exist to not check files multiple times. [...][...]
    • Add an add_comment helper method; don't mess with our internal list directly. [...]
    • Replace some simple usages of str.format with Python 'f-strings' [...] and make it easier to navigate to the main.py entry point [...].
    • In the RData comparator, always explicitly return None in the failure case as we return a non-None value in the success one. [...]
    • Tidy some imports [...][...][...] and don't alias a variable when don't end up it and use _ instead. [...]
    • Clarify the use of a separate NullChanges quasi-file to represent missing data in the Debian package comparator [...] and clarify use of a 'null' diff in order to remember an exit code. [...]
  • Misc:


§


Debian

In Debian, I made the following uploads this month:


§


Debian LTS

This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 for the Extended LTS project. This included:

You can find out more about the project via the following video:

31 July, 2020 09:55PM

François Marier

Extending GPG key expiry

Extending the expiry on a GPG key is not very hard, but it's easy to forget a step. Here's how I did my last expiry bump.

Update the expiry on the main key and the subkey:

gpg --edit-key KEYID
> expire
> key 1
> expire
> save

Upload the updated key to the keyservers:

gpg --export KEYID | curl -T - https://keys.openpgp.org
gpg --keyserver keyring.debian.org --send-keys KEYID

31 July, 2020 06:07PM

hackergotchi for Jonathan Carter

Jonathan Carter

Free Software Activities for 2020-07

Here are my uploads for the month of July, which is just a part of my free software activities, I’ll try to catch up on the rest in upcoming posts. I haven’t indulged in online conferences much over the last few months, but this month I attended the virtual editions of Guadec 2020 and HOPE 2020. HOPE isn’t something I knew about before and I enjoyed it a lot, you can find their videos on archive.org.

Debian Uploads

2020-07-05: Sponsor backport gamemode-1.5.1-5 for Debian buster-backports.

2020-07-06: Sponsor package piper (0.5.1-1) for Debian unstable (mentors.debian.net request).

2020-07-14: Upload package speedtest-cli (2.0.2-1+deb10u1) to Debian buster (Closes: #940165, #965116).

2020-07-15: Upload package calamares (3.2.27-1) to Debian unstable.

2020-07-15: Merge MR#1 for gnome-shell-extension-dash-to-panel.

2020-07-15: Upload package gnome-shell-extension-dash-to-panel (38-1) to Debian unstable.

2020-07-15: Upload package gnome-shell-extension-disconnect-wifi (25-1) to Debian unstable.

2020-07-15: Upload package gnome-shell-extension-draw-on-your-screen (6.1-1) to Debian unstable.

2020-07-15: Upload package xabacus (8.2.8-1) to Debian unstable.

2020-07-15: Upload package s-tui (1.0.2-1) to Debian unstable.

2020-07-15: Upload package calamares-settings-debian (10.0.2-1+deb10u2) to Debian buster (Closes: #934503, #934504).

2020-07-15: Upload package calamares-settings-debian (10.0.2-1+deb10u3) to Debian buster (Closes: #959541, #965117).

2020-07-15: Upload package calamares-settings-debian (11.0.2-1) to Debian unstable.

2020-07-19: Upload package bluefish (2.2.11+svn-r8872-1) to Debian unstable (Closes: #593413, #593427, #692284, #730543, #857330, #892502, #951143).

2020-07-19: Upload package bundlewrap (4.0.0-1) to Debian unstable.

2020-07-20: Upload package bluefish (2.2.11+svn-r8872-1) to Debian unstable (Closes: #965332).

2020-07-22: Upload package calamares (3.2.27-1~bpo10+1) to Debian buster-backports.

2020-07-24: Upload package bluefish (2.2.11_svn-r8872-3) to Debian unstable (Closes: #965944).

31 July, 2020 05:01PM by jonathan

July 30, 2020

Russell Coker

July 29, 2020

hackergotchi for Norbert Preining

Norbert Preining

KDE/Plasma Status Update 2020-07-30

Only a short update on the current status of my KDE/Plasma package for Debian sid and testing:

  • Frameworks 5.72
  • Plasma 5.19.4
  • Apps 20.04.3
  • Digikam 7.0.0
  • Ark CVE-2020-16116 fixed in version 20.04.3-1~np2

Hope that helps a few people. See this post for how to setup archives.

Enjoy.

29 July, 2020 11:03PM by Norbert Preining

Dima Kogan

An awk corner case?

So even after years and years of experience, core tools still find ways to surprise me. Today I tried to do some timestamp comparisons with mawk (vnl-filter, to be more precise), and ran into a detail of the language that made it not work. Not a bug, I guess, since both mawk and gawk are affected. I'll claim "language design flaw", however.

Let's say I'm processing data with unix timestamps in it (seconds since the epoch). gawk and recent versions of mawk have strftime() for that:

$ date
Wed Jul 29 15:31:13 PDT 2020

$ date +"%s"
1596061880

$ date +"%s" | mawk '{print strftime("%H",$1)}'
15

And let's say I want to do something conditional on them. I want only data after 9:00 each day:

$ date +"%s" | mawk 'strftime("%H",$1) >= 9 {print "Yep. After 9:00"}'

That's right. No output. But it is 15:31 now, and I confirmed above that strftime() reports the right time, so it should know that it's after 9:00, but it doesn't. What gives?

As we know, awk (and perl after it) treat numbers and strings containing numbers similarly: 5+5 and ="5"+5= both work the same, which is really convenient. This can only work if it can be inferred from context whether we want a number or a string; it knows that addition takes two numbers, so it knows to convert ="5"= into a number in the example above.

But what if an operator is ambiguous? Then it picks a meaning based on some internal logic that I don't want to be familiar with. And apparently awk implements string comparisons with the same < and > operators, as numerical comparisons, creating the ambiguity I hit today. strftime returns strings, and you get silent, incorrect behavior that then demands debugging. How to fix? By telling awk to treat the output of strftime() as a number:

$ date +"%s" | mawk '0+strftime("%H",$1) >= 9 {print "Yep. After 9:00"}'

Yep. After 9:00

With the benefit of hindsight, they really should not have reused any operators for both number and string operations. Then these ambiguities wouldn't occur, and people wouldn't be grumbling into their blogs decades after these decisions were made.

29 July, 2020 10:45PM by Dima Kogan

Debian Community News

Why has Debian been gripped by vendettas?

Debian Community News and the Uncensored Debian Planet site (follow the RSS feed if you want all sides of the story) have cast a new light on uncomfortable truths about the way free software is produced.

As people dig deeper, they are surprised to find that evidence of cult phenomena is indisputable while counter-accusations against victims lack any evidence whatsoever.

Nonetheless, running an elaborate cult surely takes time and effort. Why would anybody bother to do this?

Credibility of Debian's name

Debian, thanks to the the Debian Social Contract, long history and technical reliability, has built up a reputation for technical competence.

When somebody associated with Debian points out that Google's privacy policy is no more than a modern-day re-write of The Emperor's New Clothes, their concerns are often amplified and widely noticed.

Companies like Google resent this, so they exert influence in various ways to discredit those individuals who speak the truth.

We see exactly the same phenomena in the United States right now where President Trump has been trying to undermine his country's leading expert on pandemics, Dr Anthony Fauci.

Dr Anthony Fauci

Personal benefit

Earlier this month, we saw former French Prime Minister François Fillon convicted of embezzlement after creating jobs for his wife and children.

This was a reminder for many of us about the way jobs are allocated to insiders in Free Software. One of the most notable examples has been the case of the former Debian Project Leader's girlfriend leaving her job at FSF and immediately being employed by a close friend of her boyfriend who is now executive director of the GNOME Foundation. Molly de Blanc is not a programmer but she is frequently given keynote speaking opportunities in free software events. People asking how she is selected over other women and which organization pays for her travel to these events only receive vague replies.

In one event, FOSDEM, at ULB in Brussels, Miss de Blanc used her talk to humiliate other volunteers in Debian, including one who was at the event. The people she boasts about demoting have over 30 years experience between them, while de Blanc's Github page confesses ....

Molly de Blanc, Mollamby, girlfriend of Chris Lamb, Debian Project Leader, DPL, harassment

Elimination of democracy

One of the greatest fears of these oligarchs is that an independent candidate might be elected in their organization and go through the historic email records and bank statements, publishing facts about these conflicts of interest.

A range of organizations have canceled elections and eliminated voting rights for the majority of volunteers. When a Debian Developer was elected in another organization, FSFE e.V., people immediately started trying to undermine his Debian role. Linux Foundation eliminated voting rights in 2016 and FSFE e.V. canceled elections in 2018. Fedora (Red Hat) elections require candidates to submit their platforms for approval by existing management.

In parallel with these regressions, organizations have sought to denounce and villify the people who had been elected. This should be seen for what it is: an attack on an elected representative is the most vile attack on democracy.

Modern day slavery: Getting employees without a salary

Another key point is that the people being publicly shamed are all volunteers. We never see Google, Red Hat or Ubuntu publicly shaming one of their employees. In those cases where employees have been sacked, it has been done very quietly.

Yet we see these incredibly vocal shamings of people like Dr Norbert Preining and security researcher Jacob Appelbaum in Debian.

There is a pattern here: the organizations want other volunteers to see the shaming and humiliation so that everybody else will be silent and obedient like employees, but without a salary.

It all adds up

Volunteers who are not allowed to vote and not allowed to speak are getting a pretty bad deal. Even your local bridge club provides better rights and protections for members.

Think about the counterfactual: all these highly experienced engineers working together as volunteers to make genuinely secure and private alternatives to services like Gmail and Skype. It is not hard to see why companies have strong reasons to castrate free software organizations by chopping out their most talented and principled engineers.

29 July, 2020 01:00PM

Enrico Zini

Building and packaging a sysroot

This is part of a series of posts on compiling a custom version of Qt5 in order to develop for both amd64 and a Raspberry Pi.

After having had some success with a sysroot in having a Qt5 cross-build environment that includes QtWebEngine, the next step is packaging the sysroot so it can be available both to build the cross-build environment, and to do cross-development with it.

The result is this Debian source package which takes a Raspberry Pi OS disk image, provisions it in-place, extracts its contents, and packages them.

Yes. You may want to reread the last paragraph.

It works directly in the disk image to avoid a nasty filesystem issue on emulated 32bit Linux over a 64bit mounted filesystem.

This feels like the most surreal Debian package I've ever created, and this saga looks like one of the hairiest yaks I've ever shaved.

Integrating this monster codebase, full of bundled code and hacks, into a streamlined production and deployment system has been for me a full stack nightmare, and I have a renewed and growing respect for the people in the Qt/KDE team in Debian, who manage to stay on top of this mess, so that it all just works when we need it.

29 July, 2020 08:15AM

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

Installing and Running Ubuntu on a 2015-ish MacBook Air

So a few months ago kiddo one dropped an apparently fairly large cup of coffee onto her one and only trusted computer. With a few months (then) to graduation (which by now happened), and with the apparent “genuis bar” verdict of “it’s a goner” a new one was ordered. As it turns out this supposedly dead one coped well enough with the coffee so that after a few weeks of drying it booted again. But give the newer one, its apparent age and whatnot, it was deemed surplus. So I poked around a little on the interwebs and conclude that yes, this could work.

Fast forward a few months and I finally got hold of it, and had some time to play with it. First, a bootable usbstick was prepared, and the machine’s content was really (really, and check again: really) no longer needed, I got hold of it for good.

tl;dr It works just fine. It is a little heavier than I thought (and isn’t “air” supposed to be weightless?) The ergonomics seem quite nice. The keyboard is decent. Screen-resolution on this pre-retina simple Air is so-so at 1440 pixels. But battery live seems ok and e.g. the camera is way better than what I have in my trusted Lenovo X1 or at my desktop. So just as a zoom client it may make a lot of sense; otherwise just walking around with it as a quick portable machine seems perfect (especially as my Lenovo X1 still (ahem) suffers from one broken key I really need to fix…).

Below are some lightly edited notes from the installation. Initial steps were quick: maybe an hour or less? Customizing a machine takes longer than I remembered, this took a few minutes here and there quite a few times, but always incremental.

Initial Steps

  • Download of Ubuntu 20.04 LTS image: took a few moments, even on broadband, feels slower than normal (fast!) Ubuntu package updates, maybe lesser CDN or bad luck

  • Startup Disk Creator using a so-far unused 8gb usb drive

  • Plug into USB, recycle power, press “Option” on macOS keyboard: voila

  • After a quick hunch… no to ‘live/test only’ and yes to install, whole disk

  • install easy, very few questions, somehow skips wifi

  • so activate wifi manually — and everythings pretty much works

Customization

  • First deal with ‘fn’ and ‘ctrl’ key swap. Install git and followed this github repo which worked just fine. Yay. First (manual) Linux kernel module build needed need in … half a decade? Longer?

  • Fire up firefox, go to ‘download chrome’, install chrome. Sign in. Turn on syncing. Sign into Pushbullet and Momentum.

  • syncthing which is excellent. Initially via apt, later from their PPA. Spend some time remembering how to set up the mutual handshakes between devices. Now syncing desktop/server, lenovo x1 laptop, android phone and this new laptop

  • keepassx via apt and set up using Sync/ folder. Now all (encrypted) passwords synced.

  • Discovered synergy now longer really free, so after a quick search found and installed barrier (via apt) to have one keyboard/mouse from desktop reach laptop.

  • Added emacs via apt, so far ‘empty’, so config files yet

  • Added ssh via apt, need to propagate keys to github and gitlab

  • Added R via add-apt-repository --yes "ppa:marutter/rrutter4.0" and add-apt-repository --yes "ppa:c2d4u.team/c2d4u4.0+". Added littler and then RStudio

  • Added wajig (apt frontend) and byobu, both via apt

  • Created ssh key, shipped it to server and github + gitlab

  • Cloned (not-public) ‘dotfiles’ repo and linked some dotfiles in

  • Cloned git repo for nord-theme for gnome terminal and installed it; also added it to RStudio via this repo

  • Emacs installed, activated dotfiles, then incrementally install a few elpa-* packages and a few M-x package-install including nord-theme, of course

  • Installed JetBrains Mono font from my own local package; activated for Gnome Terminal and Emacs

  • Install gnome-tweak-tool via apt, adjusted a few settings

  • Ran gsettings set org.gnome.desktop.wm.preferences focus-mode 'sloppy'

  • Set up camera following this useful GH repo

  • At some point also added slack and zoom, because, well, it is 2020

  • STILL TODO:

    • docker
    • bother with email setup?,
    • maybe atom/code/…?

29 July, 2020 01:52AM

July 28, 2020

hackergotchi for Chris Lamb

Chris Lamb

Pop culture matters

Many people labour under the assumption that pop culture is trivial and useless while only 'high' art can grant us genuine and eternal knowledge about the world. Given that we have a finite time on this planet, we are all permitted to enjoy pop culture up to a certain point, but we should always minimise our interaction with it, and consume more moral and intellectual instruction wherever possible.

Or so the theory goes. What these people do not realise is that pop and mass culture can often provide more information about the world, humanity in general and — what is even more important — ourselves.

This is not quite the debate around whether high art is artistically better, simply that pop culture can be equally informative. Jeremy Bentham argued in the 1820s that "prejudice apart, the game of push-pin is of equal value with the arts and sciences of music and poetry", that it didn't matter where our pleasures come from. (John Stuart Mill, Bentham's intellectual rival, disagreed.) This fundamental question of philosophical utilitarianism will not be resolved here.

However, what might begin to be resolved is our instinctive push-back against pop culture. We all share an automatic impulse to disregard things we do not like and to pretend they do not exist, but this wishful thinking does not mean that these cultural products do not continue to exist when we aren't thinking about them and, more to our point, continue to influence others and even ourselves.

Take, for example, the recent trend for 'millennial pink'. With its empty consumerism, faux nostalgia, reductive generational stereotyping, objectively ugly æsthetics and tedious misogyny (photographed with Rose Gold iPhones), the very combination appears to have been deliberately designed to annoy me, curiously providing circumstantial evidence in favour of intelligent design. But if I were to immediately dismiss millennial pink and any of the other countless cultural trends I dislike simply because I find them disagreeable, I would be willingly keeping myself blind to their underlying ideology, their significance and their effect on society at large. If I had any ethical or political reservations I might choose not to engage with them economically or to avoid advertising them to others, but that is a different question altogether.

Even if we can't notice this pattern within ourselves we can first observe it in others. We can all recall moments where someone has brushed off a casual reference to pop culture, be it Tiger King, TikTok, team sports or Taylor Swift; if you can't, simply look for the abrupt change of tone and the slightly-too-quick dismissal. I am not suggesting you attempt to dissuade others or even to point out this mental tic, but merely seeing it in action can be highly illustrative in its own way.

In summary, we can simultaneously say that pop culture is not worthy of our time relative to other pursuits while consuming however much of it we want, but deliberately dismissing pop culture doesn't mean that a lot of other people are not interacting with it and is therefore undeserving of any inquiry. And if that doesn't convince you, just like the once-unavoidable millennial pink, simply sticking our collective heads in the sand will not mean that wider societal-level ugliness is going to disappear anytime soon.

Anyway, that's a very long way of justifying why I plan to re-watch TNG.

28 July, 2020 11:02PM

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

ttdo 0.0.6: Bugfix

A bugfix release of our (still small) ttdo package arrived on CRAN overnight. As introduced last fall, the ttdo package extends the most excellent (and very minimal / zero depends) unit testing package tinytest by Mark van der Loo with the very clever and well-done diffobj package by Brodie Gaslam to give us test results with visual diffs:

ttdo screenshot

This release corrects a minor editing error spotted by the ever-vigilant John Blischak.

The NEWS entry follow.

Changes in ttdo version 0.0.6 (2020-07-27)

  • Correct a minor editing mistake spotted by John Blischak.

CRANberries provides the usual summary of changes to the previous version. Please use the GitHub repo and its issues for any questions.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

28 July, 2020 10:36PM

hackergotchi for Jonathan Carter

Jonathan Carter

Free Software Activities for 2020-06

Hmm, this is the latest I’ve posted my monthly updates yet (nearly by a month!). June was both crazy on the incoming side, and at the same time I just wasn’t that productive (at least since then I caught up a lot). In theory, lockdown means that I spend less time in traffic, in shops or with friends and have more time to do stuff, in practice I go to bed later and later and waste more time watching tv shows and playing mobile games. A cycle that I have at least broken free from since June.

Debian Package Uploads

2020-06-04: Upload package btfs (2.21-1) to Debian unstable.

2020-06-04: Upload package gnome-shell-extension-disconnect-wifi (24-1) to Debian unstable.

2020-06-18: Sponsor package gamemode (1.5.1-5) for Debian unstable (Games team request).

2020-06-21: Upload package calamares (3.2.26-1) to Debian unstable.

2020-06-21: Upload package s-tui (1.0.1-1) to Debian unstable.

2020-06-29: Sponsor package libinih (48-1~bpo10+1) for Debian buster-backports.

2020-06-30: Upload packge calamares (3.2.26-1~bpo10+1) to Debian buster-backports.

2020-06-30: Upload package toot (0.27.0-1) to Debian unstable.

2020-06-30: Upload package calamares (3.2.26.1-1) to Debian unstable.

28 July, 2020 06:15PM by jonathan

hackergotchi for Steve Kemp

Steve Kemp

I'm a bit of a git (hacker?)

Sometimes I enjoy reading the source code to projects I like, use, or am about to install for the first time. This was something I used to do on a very regular basis, looking for security issues to report. Nowadays I don't have so much free time, but I still like to inspect the source code to new applications I install, and every now and again I'll find the time to look at the source to random projects.

Reading code is good. Reading code is educational.

One application I've looked at multiple times is redis, which is a great example of clean and well-written code. That said when reading the redis codebase I couldn't help noticing that there were a reasonably large number of typos/spelling mistakes in the comments, so I submitted a pull-request:

Sadly that particular pull-request didn't receive too much attention, although a previous one updating the configuration file was accepted. I was recently reminded of these pull-requests when I was when I was doing some other work. So I figured I'd have a quick scan of a couple of other utilities.

In the past I'd just note spelling mistakes when I came across them, usually I'd be opening each file in a project one by one and reading them from top to bottom. (Sometimes I'd just open files in emacs and run "M-x ispell-comments-and-strings", but more often I'd just notice them with my eyes). It did strike me that if I were to do this in a more serious fashion it would be good to automate it.

So this time round I hacked up a simple "dump comments" utility, which would scan named files and output the contents of any comments (be they single-line, or multi-line). Once I'd done that I could spell-check easily:

 $ go run dump-comments.go *.c > comments
 $ aspell -c comments

Anyway the upshot of that was a pull-request against git:

We'll see if that makes its way live sometime. In case I get interested in doing this again I've updated my sysbox-utility collection to have a comments sub-command. That's a little more robust and reliable than my previous hack:

$ sysbox comments -pretty=true $(find . -name '*.c')
..
..

The comments sub-command has support for:

  • Single-line comments, for C, as prefixed with //.
  • Multi-line comments, for C++, as between /* and */.
  • Single-line comments, for shell, as prefixed with #.
  • Lua comments, both single-line (prefixed with --) and multiline between --[[ and --]].

Adding new support would be trivial, I just need a start and end pattern to search against. Pull-requests welcome:

28 July, 2020 05:51PM

Russ Allbery

Review: The City in the Middle of the Night

Review: The City in the Middle of the Night, by Charlie Jane Anders

Publisher: Tor
Copyright: February 2019
Printing: February 2020
ISBN: 1-4668-7113-X
Format: Kindle
Pages: 366

January is a tidally-locked planet divided between permanent night and permanent day, an unfortunate destination for a colony starship. Now, humans cling to a precarious existence along the terminator, huddling in two wildly different cities and a handful of smaller settlements, connected by a road through the treacherous cold.

The novel opens with Sophie, a shy university student from the dark side of the city of Xiosphant. She has an overwhelming crush on Bianca, her high-class, self-confident roommate and one of the few people in her life to have ever treated her with compassion and attention. That crush, and her almost non-existent self-esteem, lead her to take the blame for Bianca's petty theft, resulting in what should have been a death sentence. Sophie survives only because she makes first contact with a native intelligent species of January, one that the humans have been hunting for food and sport.

Sadly, I think this is enough Anders for me. I've now bounced off two of her novels, both for structural reasons that I think go deeper than execution and indicate a fundamental mismatch between what Anders wants to do as an author and what I'm looking for as a reader.

I'll talk more about what this book is doing in a moment, but I have to start with Bianca and Sophie. It's difficult for me to express how much I loathed this relationship and how little I wanted to read about it. It took me about five pages to peg Bianca as a malignant narcissist and Sophie's all-consuming crush as dangerous codependency. It took the entire book for Sophie to figure out how awful Bianca is to her, during which Bianca goes through the entire abusive partner playbook of gaslighting, trivializing, contingent affection, jealous rage, and controlling behavior. And meanwhile Sophie goes back to her again, and again, and again, and again. If I hadn't been reading this book on a Kindle, I think it would have physically hit a wall after their conversation in the junkyard.

This is truly a matter of personal taste and preference. This is not an unrealistic relationship; this dynamic happens in life all too often. I'm sure there is someone for whom reading about Sophie's spectacularly poor choices is affirming or cathartic. I've not personally experienced this sort of relationship, which doubtless matters.

But having empathy for someone who is making awful and self-destructive life decisions and trusting someone they should not be trusting and who is awful to them in every way is difficult work. Sophie is the victim of Bianca's abuse, but she does so many stupid and ill-conceived things in support of this twisted relationship that I found it very difficult to not get angry at her. Meanwhile, Anders writes Sophie as so clearly fragile and uncertain and devoid of a support network that getting angry at her is like kicking a puppy. The result for me was spending nearly an entire book in a deeply unpleasant state of emotional dissonance. I may be willing to go through that for a close friend, but in a work of fiction it's draining and awful and entirely not fun.

The other viewpoint character had the opposite problem for me. Mouth starts the book as a traveling smuggler, the sole survivor of a group of religious travelers called the Citizens. She's practical, tough, and guarded. Beneath that, I think the intent was to show her as struggling to come to terms with the loss of her family and faith community. Her first goal in the book is to recover a recording of Citizen sacred scripture to preserve it and to reconnect with her past.

This sounds interesting on the surface, but none of it gelled. Mouth never felt to me like someone from a faith community. She doesn't act on Citizen beliefs to any meaningful extent, she rarely talks about them, and when she does, her attitude is nostalgia without spirituality. When Mouth isn't pursuing goals that turn out to be meaningless, she aimlessly meandered through the story. Sophie at least has agency and makes some important and meaningful decisions. Mouth is just there, even when Anders does shattering things to her understanding of her past.

Between Sophie and Bianca putting my shoulders up around my ears within the first few pages of the first chapter and failing to muster any enthusiasm for Mouth, I said the eight deadly words ("I don't care what happens to these people") about a hundred pages in and the book never recovered.

There are parts of the world-building I did enjoy. The alien species that Sophie bonds with is not stunningly original, but it's a good (and detailed) take on one of the alternate cognitive and social models that science fiction has dreamed up. I was comparing the strangeness and dislocation unfavorably to China Miéville's Embassytown while I was reading it, but in retrospect Anders's treatment is more decolonialized. Xiosphant's turn to Circadianism as their manifestation of order is a nicely understated touch, a believable political overreaction to the lack of a day/night cycle. That touch is significantly enhanced by Sophie's time working in a salon whose business model is to help Xiosphant residents temporarily forget about time. And what glimmers we got of politics on the colony ship and their echoing influence on social and political structures were intriguing.

Even with the world-building, though, I want the author to be interested in and willing to expand the same bits of world-building that I'm engaged with. Anders didn't seem to be. The reader gets two contrasting cities along a road, one authoritarian and one libertine, which makes concrete a metaphor for single-axis political classification. But then Anders does almost nothing with that setup; it's just the backdrop of petty warlord politics, and none of the political activism of Bianca's student group seems to have relevance or theoretical depth. It's a similar shallowness as the religion of Mouth's Citizens: We get a few fragments of culture and religion, but without narrative exploration and without engagement from any of the characters. The way the crew of the Mothership was assembled seems to have led to a factional and racial caste system based on city of origin and technical expertise, but I couldn't tell you more than that because few of the characters seem to care. And so on.

In short, the world-building that I wanted to add up to a coherent universe that was meaningful to the characters and to the plot seemed to be little more than window-dressing. Anders tosses in neat ideas, but they don't add up to anything. They're just background scenery for Bianca and Sophie's drama.

The one thing that The City in the Middle of the Night does well is Sophie's nervous but excited embrace of the unknown. It was delightful to see the places where a typical protagonist would have to overcome a horror reaction or talk themselves through tradeoffs and where Sophie's reaction was instead "yes, of course, let's try." It provided an emotional strength to an extended first-contact exploration scene that made it liberating and heart-warming without losing the alienness. During that part of the book (in which, not coincidentally, Bianca does not appear), I was able to let my guard down and like Sophie for the first time, and I suspect that was intentional on Anders's part.

But, overall, I think the conflict between Anders's story-telling approach and my preferences as a reader are mostly irreconcilable. She likes to write about people who make bad decisions and compound their own problems. In one of the chapters of her non-fiction book about writing that's being serialized on Tor.com she says "when we watch someone do something unforgivable, we're primed to root for them as they search desperately for an impossible forgiveness." This is absolutely not true for me; when I watch a character do something unforgivable, I want to see repudiation from the protagonists and ideally some clear consequences. When that doesn't happen, I want to stop reading about them and find something more enjoyable to do with my time. I certainly don't want to watch a viewpoint character insist that the person who is doing unforgivable things is the center of her life.

If your preferences on character and story arc are closer to Anders's than mine, you may like this book. Certainly lots of people did; it was nominated for multiple awards and won the Locus Award for Best Science Fiction Novel. But despite the things it did well, I had a truly miserable time reading it and am not anxious to repeat the experience.

Rating: 4 out of 10

28 July, 2020 03:49AM

July 27, 2020

hackergotchi for Wouter Verhelst

Wouter Verhelst

On Statements, Facts, Hypotheses, Science, Religion, and Opinions

The other day, we went to a designer's fashion shop whose owner was rather adamant that he was never ever going to wear a face mask, and that he didn't believe the COVID-19 thing was real. When I argued for the opposing position, he pretty much dismissed what I said out of hand, claiming that "the hospitals are empty dude" and "it's all a lie". When I told him that this really isn't true, he went like "well, that's just your opinion". Well, no -- certain things are facts, not opinions. Even if you don't believe that this disease kills people, the idea that this is a matter of opinion is missing the ball by so much that I was pretty much stunned by the level of ignorance.

His whole demeanor pissed me off rather quickly. While I disagree with the position that it should be your decision whether or not to wear a mask, it's certainly possible to have that opinion. However, whether or not people need to go to hospitals is not an opinion -- it's something else entirely.

After calming down, the encounter got me thinking, and made me focus on something I'd been thinking about before but hadn't fully forumlated: the fact that some people in this world seem to misunderstand the nature of what it is to do science, and end up, under the claim of being "sceptical", with various nonsense things -- see scientology, flat earth societies, conspiracy theories, and whathaveyou.

So, here's something that might (but probably won't) help some people figuring out stuff. Even if it doesn't, it's been bothering me and I want to write it down so it won't bother me again. If you know all this stuff, it might be boring and you might want to skip this post. Otherwise, take a deep breath and read on...

Statements are things people say. They can be true or false; "the sun is blue" is an example of a statement that is trivially false. "The sun produces light" is another one that is trivially true. "The sun produces light through a process that includes hydrogen fusion" is another statement, one that is a bit more difficult to prove true or false. Another example is "Wouter Verhelst does not have a favourite color". That happens to be a true statement, but it's fairly difficult for anyone that isn't me (or any one of the other Wouters Verhelst out there) to validate as true.

While statements can be true or false, combining statements without more context is not always possible. As an example, the statement "Wouter Verhelst is a Debian Developer" is a true statement, as is the statement "Wouter Verhelst is a professional Volleybal player"; but the statement "Wouter Verhelst is a professional Volleybal player and a Debian Developer" is not, because while I am a Debian Developer, I am not a professional Volleybal player -- I just happen to share a name with someone who is.

A statement is never a fact, but it can describe a fact. When a statement is a true statement, either because we trivially know what it states to be true or because we have performed an experiment that proved beyond any possible doubt that the statement is true, then what the statement describes is a fact. For example, "Red is a color" is a statement that describes a fact (because, yes, red is definitely a color, that is a fact). Such statements are called statements of fact. There are other possible statements. "Grass is purple" is a statement, but it is not a statement of fact; because as everyone knows, grass is (usually) green.

A statement can also describe an opinion. "The Porsche 911 is a nice car" is a statement of opinion. It is one I happen to agree with, but it is certainly valid for someone else to make a statement that conflicts with this position, and there is nothing wrong with that. As the saying goes, "opinions are like assholes: everyone has one". Statements describing opinions are known as statements of opinion.

The differentiating factor between facts and opinions is that facts are universally true, whereas opinions only hold for the people who state the opinion and anyone who agrees with them. Sometimes it's difficult or even impossible to determine whether a statement is true or not. The statement "The numbers that win the South African Powerball lottery on the 31st of July 2020 are 2, 3, 5, 19, 35, and powerball 14" is not a statement of fact, because at the time of writing, the 31st of July 2020 is in the future, which at this point gives it a 1 in 24,435,180 chance to be true). However, that does not make it a statement of opinion; it is not my opinion that the above numbers will win the South African powerball; instead, it is my guess that those numbers will be correct. Another word for "guess" is hypothesis: a hypothesis is a statement that may be universally true or universally false, but for which the truth -- or its lack thereof -- cannot currently be proven beyond doubt. On Saturday, August 1st, 2020 the above statement about the South African Powerball may become a statement of fact; most likely however, it will instead become a false statement.

An unproven hypothesis may be expressed as a matter of belief. The statement "There is a God who rules the heavens and the Earth" cannot currently (or ever) be proven beyond doubt to be either true or false, which by definition makes it a hypothesis; however, for matters of religion this is entirely unimportant, as for believers the belief that the statement is correct is all that matters, whereas for nonbelievers the truth of that statement is not at all relevant. A belief is not an opinion; an opinion is not a belief.

Scientists do not deal with unproven hypotheses, except insofar that they attempt to prove, through direct observation of nature (either out in the field or in a controlled laboratory setting) that the hypothesis is, in fact, a statement of fact. This makes unprovable hypotheses unscientific -- but that does not mean that they are false, or even that they are uninteresting statements. Unscientific statements are merely statements that science cannot either prove or disprove, and that therefore lie outside of the realm of what science deals with.

Given that background, I have always found the so-called "conflict" between science and religion to be a non-sequitur. Religion deals in one type of statements; science deals in another. The do not overlap, since a statement can either be proven or it cannot, and religious statements by their very nature focus on unprovable belief rather than universal truth. Sure, the range of things that science has figured out the facts about has grown over time, which implies that religious statements have sometimes been proven false; but is it heresy to say that "animals exist that can run 120 kph" if that is the truth, even if such animals don't exist in, say, Rome?

Something very similar can be said about conspiracy theories. Yes, it is possible to hypothesize that NASA did not send men to the moon, and that all the proof contrary to that statement was somehow fabricated. However, by its very nature such a hypothesis cannot be proven or disproven (because the statement states that all proof was fabricated), which therefore implies that it is an unscientific statement.

It is good to be sceptical about what is being said to you. People can have various ideas about how the world works, but only one of those ideas -- one of the possible hypotheses -- can be true. As long as a hypothesis remains unproven, scientists love to be sceptical themselves. In fact, if you can somehow prove beyond doubt that a scientific hypothesis is false, scientists will love you -- it means they now know something more about the world and that they'll have to come up with something else, which is a lot of fun.

When a scientific experiment or observation proves that a certain hypothesis is true, then this probably turns the hypothesis into a statement of fact. That is, it is of course possible that there's a flaw in the proof, or that the experiment failed (but that the failure was somehow missed), or that no observance of a particular event happened when a scientist tried to observe something, but that this was only because the scientist missed it. If you can show that any of those possibilities hold for a scientific proof, then you'll have turned a statement of fact back into a hypothesis, or even (depending on the exact nature of the flaw) into a false statement.

There's more. It's human nature to want to be rich and famous, sometimes no matter what the cost. As such, there have been scientists who have falsified experimental results, or who have claimed to have observed something when this was not the case. For that reason, a scientific paper that gets written after an experiment turned a hypothesis into fact describes not only the results of the experiment and the observed behavior, but also the methodology: the way in which the experiment was run, with enough details so that anyone can retry the experiment.

Sometimes that may mean spending a large amount of money just to be able to run the experiment (most people don't have an LHC in their backyard, say), and in some cases some of the required materials won't be available (the latter is expecially true for, e.g., certain chemical experiments that involve highly explosive things); but the information is always there, and if you spend enough time and money reading through the available papers, you will be able to independently prove the hypothesis yourself. Scientists tend to do just that; when the results of a new experiment are published, they will try to rerun the experiment, partially because they want to see things with their own eyes; but partially also because if they can find fault in the experiment or the observed behavior, they'll have reason to write a paper of their own, which will make them a bit more rich and famous.

I guess you could say that there's three types of people who deal with statements: scientists, who deal with provable hypotheses and statements of fact (but who have no use for unprovable hypotheses and statements of opinion); religious people and conspiracy theorists, who deal with unprovable hypotheses (where the religious people deal with these to serve a large cause, while conspiracy theorists only care about the unprovable hypotheses); and politicians, who should care about proven statements of fact and produce statements of opinion, but who usually attempt the reverse of those two these days :-/

Anyway...

mic drop

27 July, 2020 04:00PM

hackergotchi for Steve Kemp

Steve Kemp

Growing food is fun.

"I grew up on a farm" is something I sometimes what I tell people. It isn't true, but it is a useful shorthand. What is true is that my parents both come from a farming background, my father's family up in Scotland, my mother's down in Yorkshire.

Every summer my sisters and myself would have a traditional holiday at the seaside, which is what people do in the UK (Blackpool, Scarborough, Great Yarmouth, etc). Before, or after, that we'd spend the rest of the summer living on my grandmother's farm.

I loved spending time on the farm when I was a kid, and some of my earliest memories date from that time. For example I remember hand-feeding carrots to working dogs (alsatians) that were taller than I was. I remember trying to ride on the backs of those dogs, and how that didn't end well. In fact the one and only time I can recall my grandmother shouting at me, or raising her voice at all, was when my sisters and I spent an afternoon playing in the coal-shed. We were filthy and covered in coal-dust from head to toe. Awesome!

Anyway the only reason I bring this up is because I have a little bit of a farming background, largely irrelevant in my daily life, but also a source of pleasant memories. Despite it being an animal farm (pigs, sheep, cows) there was also a lot of home-grown food, which my uncle Albert would deliver/sell to people nearby out of the back of a van. That same van that would be used to ferry us to see the fireworks every November. Those evenings were very memorable too - they would almost always involve flasks of home-made vegetable soup.

Nowadays I live in Finland, and earlier in the year we received access to an allotment - a small piece of land (10m x 10m) for €50/year - upon which we can grow our own plants, etc.

My wife decided to plant flowers and make it look pretty. She did good.

I decided to plant "food". I might not have done this stuff from scratch before, but I was pretty familiar with the process from my youth, and also having the internet to hand to make the obvious searches such as "How do you know when you can harvest your garlic?"

Before I started I figured it couldn't be too hard, after all if you leave onions/potatoes in the refrigerator for long enough they start to grow! It isn't like you have to do too much to help them. In short it has been pretty easy and I'm definitely going to be doing more of it next year.

I've surprised myself by enjoying the process as much as I have. Every few days I go and rip up the weeds, and water the things we've planted. So far I've planted, and harvested, Radish, Garlic, Onions, and in a few more weeks I'll be digging up potatoes.

I have no particular point to this post, except to say that if you have a few hours spare a week, and a slab of land to hand upon which you can dig and plant I'd recommend it. Sure there were annoyances, and not a single one of the carrot-seeds I planted showed any sign of life, but the other stuff? The stuff that grew? Very tasty, om nom nom ..

(It has to be said that when we received the plot there was a jungle growing upon it. Once we tidied it all up we found raspberries, roses, and other things. The garlic I reaped was already growing so I felt like a cheat to harvest it. That said I did plant a couple of bulbs on my balcony so I could say "I grew this from scratch". Took a while, but I did indeed harvest my own garlic.)

27 July, 2020 12:00PM

hackergotchi for Martin Michlmayr

Martin Michlmayr

ledger2beancount 2.4 released

I released version 2.4 of ledger2beancount, a ledger to beancount converter.

There are two notable changes in this release:

  1. I fixed two regressions introduced in the last release. Sorry about the breakage!
  2. I improved support for hledger. I believe all syntax differences in hledger are supported now.

Here are the changes in 2.4:

  • Fix regressions introduced in version 2.3
    • Handle price directives with comments
    • Don't assume implicit conversion when price is on second posting
  • Improve support for hledger
    • Fix parsing of hledger tags
    • Support commas as decimal markers
    • Support digit group marks through commodity and D directives
    • Support end aliases directive
    • Support regex aliases
    • Recognise total balance assertions
    • Recognise sub-account balance assertions
  • Add support for define directive
  • Convert all uppercase metadata tags to all lowercase
  • Improve handling of ledger lots without cost
  • Allow transactions without postings
  • Fix parsing issue in commodity declarations
  • Support commodities that contain quotation marks
  • Add --version option to show version
  • Document problem of mixing apply and include

Thanks to Kirill Goncharov for pointing out one regressions, to Taylor R Campbell for for a patch, to Stefano Zacchiroli for some input, and finally to Simon Michael for input on hledger!

You can get ledger2beancount from GitHub

27 July, 2020 08:07AM by Martin Michlmayr

Russ Allbery

Summer haul

I'm buying rather too many books at the moment and not reading enough of them (in part because I got back into Minecraft and in part because I got a bit stuck on a few difficult books). I think I've managed to get myself unstuck again, though, and have started catching up on reviews.

2020. It's kind of a lot. And I'm not even that heavily affected.

Katherine Addison — The Angel of the Crows (sff)
Marie Brennan — A Natural History of Dragons (sff)
Kacen Callender — Queen of the Conquered (sff)
Jo Clayton — Diadem from the Stars (sff)
Jo Clayton — Lamarchos (sff)
Jo Clayton — Irsud (sff)
Clifford D. Conner — The Tragedy of American Science (nonfiction)
Kate Elliott — Unconquerable Sun (sff)
Rory Fanning & Craig Hodges — Long Shot (nonfiction)
Michael Harrington — Socialism: Past & Future (nonfiction)
Nalo Hopkinson — Brown Girl in the Ring (sff)
Kameron Hurley — The Stars Are Legion (sff)
N.K. Jemisin — Emergency Skin (sff)
T. Kingfisher — A Wizard's Guide to Defensive Baking (sff)
T. Kingfisher — Nine Goblins (sff)
Michael Lewis — The Fifth Risk (nonfiction)
Paul McAuley — War of the Maps (sff)
Gretchen McCulloch — Because Internet (nonfiction)
Hayao Miyazaki — Nausicaä of the Valley of the Wind (graphic novel)
Annalee Newitz — The Future of Another Timeline (sff)
Nick Pettigrew — Anti-Social (nonfiction)
Rivers Solomon, et al. — The Deep (sff)
Jo Walton — Or What You Will (sff)
Erik Olin Wright — Stardust to Stardust (nonfiction)

Of these, I've already read and reviewed The Fifth Risk (an excellent book).

27 July, 2020 04:31AM

Review: Rise of the Warrior Cop

Review: Rise of the Warrior Cop, by Radley Balko

Publisher: PublicAffairs
Copyright: 2013
ISBN: 1-61039-212-4
Format: Kindle
Pages: 336

As the United States tries, in fits and starts, to have a meaningful discussion about long-standing police racism, brutality, overreach, corruption, and murder, I've realized that my theoretical understanding of the history of and alternative frameworks for law enforcement is woefully lacking. Starting with a book by a conservative white guy is not the most ideal of approaches, but it's what I already had on hand, and it won't be the last book I read and review on this topic. (Most of my research so far has been in podcast form. I don't review those here, but I can recommend Ezra Klein's interviews with Ta-Nehisi Coates, Paul Butler, and, most strongly, sujatha baliga.)

Rise of the Warrior Cop is from 2013 and has had several moments of fame, no doubt helped by Balko's connections to the conservative and libertarian right. One of the frustrating facts of US politics is that critiques of the justice system from the right (and from white men) get more media attention than critiques from the left. That said, it's a generally well-respected book on the factual history of the topic, and police brutality and civil rights are among the points on which I have stopped-clock agreements with US libertarians.

This book is very, very libertarian.

In my callow youth, I was an ardent libertarian, so I've read a lot of US libertarian literature. It's a genre with its own conventions that become obvious when you read enough of it, and Rise of the Warrior Cop goes through them like a checklist. Use the Roman Republic (never the Roman Empire) as the starting point for any political discussion, check. Analyze the topic in the context of pre-revolutionary America, check. Spend considerable effort on discerning the opinions of the US founders on the topic since their opinions are always relevant to the modern world, check. Locate some point in the past (preferably before 1960) where the political issue was as good as it has ever been, check. Frame all changes since then as an erosion of rights through government overreach, check. Present your solution as a return to a previous era of respect for civil rights, check. Once you start recognizing the genre conventions, their prevalence in libertarian writing is almost comical.

The framing chapters therefore leave a bit to be desired, but the meat of the book is a useful resource. Starting with the 1970s and its use as a campaigning tool by Nixon, Balko traces a useful history of the war on drugs. And starting with the 1980s, the number of cites to primary sources and the evidence of Balko's own research increases considerably. If you want to know how US police turned into military cosplayers with body armor, heavy weapons, and armored vehicles, this book provides a lot of context and history.

One of the reasons why I view libertarians as allies of convenience on this specific issue is that drug legalization and disgust with the war on drugs have been libertarian issues for decades. Ideologically honest libertarians (and Balko appears to be one) are inherently skeptical of the police, so when the police overreach in an area of libertarian interest, they notice. Balko makes a solid argument, backed up with statistics, specific programs, legislation, and court cases, that the drug war and its accompanying lies about heavily-armed drug dealers and their supposed threat to police officers was the fuel for the growth of SWAT teams, no-knock search warrants, erosion of legal protections for criminal defendants, and de facto license for the police to ignore the scope and sometimes even the existence of warrants.

This book is useful support for the argument that fears for the safety of officers underlying the militarization of police forces are imaginary. One telling point that Balko makes repeatedly and backs with statistical and anecdotal evidence is that the police generally do not use raid tactics on dangerous criminals. On the contrary, aggressive raids are more likely to be used on the least dangerous criminals because they're faster, they're fun for the police (they provide an adrenaline high and let them play with toys), and they're essentially risk-free. If the police believe someone is truly dangerous, they're more likely to use careful surveillance and to conduct a quiet arrest at an unexpected moment. The middle-of-the-night armed break-ins with battering rams, tear gas, and flash-bangs are, tellingly, used against the less dangerous suspects.

This is part of Balko's overall argument that police equipment and tactics have become untethered from any realistic threat and have become cultural. He traces an acceleration of that trend to 9/11 and the resulting obsession with terrorism, which further opened the spigot of military hardware and "special forces" training. This became a point of competition between police departments, with small town forces that had never seen a terrorist and had almost no chance of a terrorist incident demanding their own armored vehicles. I've encountered this bizarre terrorism justification personally; one of the reasons my local police department gave in a public hearing for not having a policy against shooting at moving vehicles was "but what if terrorism?" I don't believe there has ever been a local terrorist attack.

SWAT in such places didn't involve the special training or dedicated personnel of large city forces; instead, it was a part-time duty for normal police officers, and frequently they were encouraged to practice SWAT tactics by using them at random for some otherwise normal arrest or search. Balko argues that those raids were more exciting than normal police work, leading to a flood of volunteers for that duty and a tendency to use them as much as possible. That in turn normalizes disconnecting police tactics from the underlying crime or situational risk.

So far, so good. But despite the information I was able to extract from it, I have mixed feelings about Rise of the Warrior Cop as a whole. At the least, it has substantial limitations.

First, I don't trust the historical survey of policing in this book. Libertarian writing makes for bad history. The constraints of the genre require overusing only a few points of reference, treating every opinion of the US founders as holy writ, and tying forward progress to a return to a previous era, all of which interfere with good analysis. Balko also didn't do the research for the historical survey, as is clear from the footnotes. The citations are all to other people's histories, not to primary sources. He's summarizing other people's histories, and you'll almost certainly get better history by finding well-respected historians who cover the same ground. (That said, if you're not familiar with Peel's policing principles, this is a good introduction.)

Second, and this too is unfortunately predictable in a libertarian treatment, race rarely appears in this book. If Balko published the same book today, I'm sure he would say more about race, but even in 2013 its absence is strange. I was struck while reading by how many examples of excessive police force were raids on west coast pot farms; yes, I'm sure that was traumatic, but it's not the demographic I would name as the most vulnerable to or affected by police brutality. West coast pot growers are, however, mostly white.

I have no idea why Balko made that choice. Perhaps he thought his target audience would be more persuaded by his argument if he focused on white victims. Perhaps he thought it was an easier and less complicated story to tell. Perhaps, like a lot of libertarians, he doesn't believe racism has a significant impact on society because it would be a market failure. Perhaps those were the people who more readily came to mind. But to talk about police militarization, denial of civil rights, and police brutality in the United States without putting race at the center of both the history and the societal effects leaves a gaping hole in the analysis.

Given that lack of engagement, I also am dubious of Balko's policy prescriptions. His reform suggestions aren't unreasonable, but they stay firmly in the centrist and incrementalist camp and would benefit white people more than black people. Transparency, accountability, and cultural changes are all fine and good, but the cultural change Balko is focused on is less aggressive arrest tactics, more use of mediation, and better physical fitness. I would not object to those things (well, maybe the last, which seemed odd), but we need to have a discussion about police white supremacist organizations, the prevalence of spousal abuse, and the police tendency to see themselves not as public servants but as embattled warriors who are misunderstood by the naive sheep they are defending.

And, of course, you won't find in Rise of the Warrior Cop any thoughtful wrestling with whether there are alternative approaches to community safety, whether punitive rather than restorative justice is effective, or whether crime is a symptom of deeper societal problems we could address but refuse to. The most radical suggestion Balko has is to legalize drugs, which is both the predictable libertarian position and, as we have seen from recent events in the United States, far from the only problem of overcriminalization.

I understand why this book is so frequently mentioned on-line, and its author's political views may make it more palatable to some people than a more race-centered or radical perspective. But I don't think this is the best or most useful book on police violence that one could read today. I hope to find a better one in upcoming reviews.

Rating: 6 out of 10

27 July, 2020 02:20AM

July 26, 2020

Enrico Zini

Russ Allbery

Review: Paladin's Grace

Review: Paladin's Grace, by T. Kingfisher

Publisher: Red Wombat Studio
Copyright: 2020
ASIN: B0848Q8JVW
Format: Kindle
Pages: 399

Stephen was a paladin. Then his god died.

He was a berserker, an unstoppable warrior in the service of his god. Now, well, he's still a berserker, but going berserk when you don't have a god to control the results is not a good idea. He and his brothers were taken in by the Temple of the Rat, where they serve as guards, watch out for each other, and try to get through each day with an emptiness in their souls where a god should be.

Stephen had just finished escorting a healer through some of the poorer parts of town when a woman runs up to him and asks him to hide her. Their awkward simulated tryst is sufficient to fool the two Motherhood priests who were after her for picking flowers from the graveyard. Stephen then walks her home and that would have been the end of it, except that neither could get the other out of their mind.

Despite first appearances, and despite being set in the same world and sharing a supporting character, this is not the promised sequel to Swordheart (which is apparently still coming). It's an entirely different paladin story. T. Kingfisher (Ursula Vernon's nom de plume when writing for adults) has a lot of things to say about paladins! And, apparently, paladin-involved romances.

On the romance front, Kingfisher clearly has a type. The general shape of the story will be familiar from Swordheart and The Wonder Engine: An independent and occasionally self-confident woman with various quirks, a hunky paladin who is often maddeningly dense, and a lot of worrying on both sides about whether the other person is truly interested in them and if their personal liabilities make a relationship a horrible idea. This is not my preferred romance formula (it provokes the occasional muttered "for the love of god just talk to each other"), but I liked this iteration of it better than the previous two, mostly because of Grace.

Grace is a perfumer, a trade she went into by being picked out of a lineup of orphans by a master perfumer for her sense of smell. One of Kingfisher's strengths as a writer is showing someone get lost in their routine day-to-day competence. When mixed with an inherently fascinating profession, this creates a great reading experience. Grace is also an abuse survivor, which made the communication difficulties with Stephen more interesting and subtle. Grace has created space and a life for herself, and her unwillingness to take risks on changes is a deep part of her sense of self and personal safety. As her past is slowly revealed, Kingfisher puts the reader in a position to share Stephen's anger and protectiveness, but then consistently puts Grace's own choices, coping mechanisms, and irritated refusal to be protected back into the center of the story. She has to accept some help as she gets entangled in the investigation of a highly political staged assassination attempt, but both that help and the relationship come on her own terms. It's very well-done.

The plot was enjoyable enough, although it involved a bit too much of constantly rising stakes and turns for the worst for my taste, and the ending had a touch of deus ex machina. Like Kingfisher's other books, though, the delight is in the unexpected details. Stephen knitting socks. Grace's frustrated obsession with why he smells like gingerbread. The beautifully practical and respectful relationship between the Temple of the Rat and Stephen's band of former paladins. (After only two books in which they play a major role, the Temple of the Rat is already one of my favorite fantasy religions.) Everything about Bishop Beartongue. Grace's friend Marguerite. And a truly satisfying ending.

The best part of this book, though, is the way Grace is shown as a complete character in a way that even most books with well-rounded characterization don't manage. Some things she does make the reader's heart ache because of the hints they provide about her past, but they're also wise and effective safety mechanisms for her. Kingfisher gives her space to be competent and prickly and absent-minded. She has a complete life: friends, work, goals, habits, and little rituals. Grace meets someone and falls in love, but one can readily imagine her not falling in love and going on with her life and that result wouldn't be tragic. In short, she feels like a grown adult who has made her own peace with where she came from and what she is doing. The book provides her an opportunity for more happiness and more closure without undermining her independence. I rarely see this in a novel, and even more rarely done this well.

If you haven't read any of Kingfisher's books and are in the mood for faux-medieval city romance involving a perfumer and a bit of political skulduggery, this is a great place to start. If you liked Swordheart, you'll probably like Paladin's Grace; like me, you may even like it a bit more. Recommended, particularly if you want something light and heart-warming.

Rating: 8 out of 10

26 July, 2020 04:25AM

July 25, 2020

Niels Thykier

Support for Debian packaging files in IDEA (IntelliJ/PyCharm)

I have been using the community editions of IntelliJ and PyCharm for a while now for Python or Perl projects. But it started to annoy me that for Debian packaging bits it would “revert” into a fancy version of notepad. Being fed up with it, I set down and spent the last week studying how to write a plugin to “fix” this.

After a few prototypes, I have now released IDEA-debpkg v0.0.3 (Link to JetBrain’s official plugin listing with screenshots). It provides a set of basic features for debian/control like syntax highlighting, various degree of content validation, folding of long fields, code completion and “CTRL + hover” documentation. For debian/changelog, it is mostly just syntax highlighting with a bit of fancy linking for now. I have not done anything for debian/rules as I noted there is a Makefile plugin, which will have to do for now.

The code is available from github and licensed under Apache-2.0. Contributors, issues/feature requests and pull requests are very welcome. Among things I could help with are:

  • Icons – both for the plugin and for the file types. Currently it is just colored text, which is as far as my artistic skills got with the space provided.
  • Color and text formatting for syntax highlighting.
  • Reports of papercut or features that would be very useful to prioritize.
  • Review of the “CTRL + hover” documentation. I am hoping for something that is help for new contributors but I am very unlikely to have gotten it right (among other because I wrote most of it to “get it done” rather than “getting it right”)

I hope you will take it for spin if you have been looking for a bit of Debian packaging support to your PyCharm or other IDEA IDE. 🙂 Please do file bugs/issues if you run into issues, rough edges or unhelpful documentation, etc.

25 July, 2020 04:28PM by Niels Thykier

July 24, 2020

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

anytime 0.3.8: Minor Maintenance

A new minor release of the anytime package arrived on CRAN overnight. This is the nineteenth release, and it comes just over six months after the previous release giving further indicating that we appear to have reached a nice level of stability.

anytime is a very focused package aiming to do just one thing really well: to convert anything in integer, numeric, character, factor, ordered, … format to either POSIXct or Date objects – and to do so without requiring a format string. See the anytime page, or the GitHub README.md for a few examples.

This release mostly plays games with CRAN. Given the lack of specification for setups on their end, reproducing test failures remains, to put it mildly, “somewhat challenging”. So we eventually gave up—and weaponed up once more and now explicitly test for the one distribution where tests failed (when they clearly passed everywhere else). With that we now have three new logical predicates for various Linux distribution flavours, and if that dreaded one is seen in one test file the test is skipped. And with that we now score twelve out of twelve OKs. This being a game of cat and mouse, I am sure someone somewhere will soon invent a new test…

The full list of changes follows.

Changes in anytime version 0.3.8 (2020-07-23)

  • A small utility function was added to detect the Linux distribution used in order to fine-tune tests once more.

  • Travis now uses Ubuntu 'bionic' and R 4.0.*.

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the anytime page. The issue tracker tracker off the GitHub repo can be use for questions and comments.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

24 July, 2020 08:11PM

hackergotchi for Mike Gabriel

Mike Gabriel

Ayatana Indicators / IDO - Menu Rendering Fixed with vanilla GTK-3+

At DebConf 17 in Montreal, I gave a talk about Ayatana Indicators [1] and the project's goal to continue the — by then already dropped out of maintenance — Ubuntu Indicators in a separate upstream project, detached from Ubuntu and its Ubuntu'isms.

Stalling

The whole Ayatana Indicators project received a bit of a show stopper by the fact that the IDO (Indicator Display Object) rendering was not working in vanilla GTK-3 without a certain patch [2] that only Ubuntu has in their GTK-3 package. Addressing GTK developers upstream some years back (after GTK 3.22 had already gone into long term maintenance mode) and asking for a late patch acceptance did not work out (as already assumed). Ayatana Indicators stalled at a level of 90% actually working fine, but those nice and shiny special widgets, like the calendar widget, the audio volume slider widgets, switch widgets, etc. could not be rendered appropriately in GTK based desktop environments (e.g. via MATE Indicator Applet) on other distros than Ubuntu.

I never really had the guts to sit down without a defined ending and find a patch / solution to this nasty problem. Ayatana Indicators stalled as a whole. I kept it alive and defended its code base against various GLib and what-not deprecations and kept it in Debian, but the software was actually partially broken / dysfunctional.

Taking the Dog for a Walk and then It Became all Light+Love

Several days back, I received a mail from Robert Tari [3]. I was outside on a hike with our dog and thought, ah well, let's check emails... I couldn't believe what I read then, 15 seconds later. I could in fact, hardly breathe...

I have known Robert from earlier email exchanges. Robert maintains various "little" upstream projects, like e.g. Caja Rename, Odio, Unity Mail, etc. that I have looked into earlier regarding Debian packaging. Robert is also a Manjaro contributor and he has been working on bringing Ayatana Indicators to Manjaro MATE. In the early days, without knowing Robert, I even forked one of his projects (indicator-notification) and turned it into an Ayatana Indicator.

Robert and I also exchanged some emails about Ayatana Indicators already a couple of weeks ago. I got the sense of him maybe being up to something already then. Oh, yeah!!!

It turned out that Robert and I share the same "love" for the Ubuntu Indicators concept [4]. From his email, it became clear that Robert had spent the last 1-2 weeks drowned in the Ayatana IDO and libayatana-indicator code and worked him self through the bowels of it in order to understand the code concept of Indicators to its very depth.

When emerging back from his journey, he presented me (or rather: the world) a patch [5] against libayatana-indicator that makes it possible to render IDO objects even if a vanilla GTK-3 is installed on the system. This patch is a game changer for Indicator lovers.

When Robert sent me his mail pointing me to this patch, I think, over the past five years, I have never felt more excited (except from the exact moment of getting married to my wife two-to-three years ago) than during that moment when my brain tried to process his email. "Like a kid on Christmas Eve...", Robert wrote in one of his later mails to me. Indeed, like a "kid on Christmas Eve", Robert.

Try It Out

As a proof of all this to the Debian people, I have just done the first release of ayatana-indicator-datetime and uploaded it to Debian's NEW queue. Robert is doing the same for Manjaro. The Ayatana Indicator Sound will follow after my vacation.

For fancy widget rendering in Ayatana Indicator's system indicators, make sure you have libayatana-indicator 0.7.0 or newer installed on your system.

Credits

One of the biggest thanks ever I send herewith to Robert Tari! Robert is now co-maintainer of Ayatana Indicators. Welcome! Now, there is finally a team of active contributors. This is so delightful!!!

References

P.S.

Expect more Ayatana Indicators to appear in your favourite distro soon...

24 July, 2020 01:56PM by sunweaver

hackergotchi for Rapha&#235;l Hertzog

Raphaël Hertzog

The Debian Handbook has been updated for Debian 10

Better late than never as we say… thanks to the work of Daniel Leidert and Jorge Maldonado Ventura, we managed to complete the update of my book for Debian 10 Buster.

You can get the electronic version on debian-handbook.info or the paperback version on lulu.com. Or you can just read it online.

Translators are busy updating their translations, with German and Norvegian Bokmal leading the way…

One comment | Liked this article? Click here. | My blog is Flattr-enabled.

24 July, 2020 10:39AM by Raphaël Hertzog

hackergotchi for Evgeni Golov

Evgeni Golov

Building documentation for Ansible Collections using antsibull

In my recent post about building and publishing documentation for Ansible Collections, I've mentioned that the Ansible Community is currently in the process of making their build tools available as a separate project called antsibull instead of keeping them in the hacking directory of ansible.git.

I've also said that I couldn't get the documentation to build with antsibull-docs as it wouldn't support collections yet. Thankfully, Felix Fontein, one of the maintainers of antsibull, pointed out that I was wrong and later versions of antsibull actually have partial collections support. So I went ahead and tried it again.

And what should I say? Two bug reports by me and four patches by Felix Fontain later I can use antsibull-docs to generate the Foreman Ansible Modules documentation!

Let's see what's needed instead of the ugly hack in detail.

We obviously don't need to clone ansible.git anymore and install its requirements manually. Instead we can just install antsibull (0.17.0 contains all the above patches). We also need Ansible (or ansible-base) 2.10 or never, which currently only exists as a pre-release. 2.10 is the first version that has an ansible-doc that can list contents of a collection, which antsibull-docs requires to work properly.

The current implementation of collections documentation in antsibull-docs requires the collection to be installed as in "Ansible can find it". We had the same requirement before to find the documentation fragments and can just re-use the installation we do for various other build tasks in build/collection and point at it using the ANSIBLE_COLLECTIONS_PATHS environment variable or the collections_paths setting in ansible.cfg1. After that, it's only a matter of passing --use-current to make it pick up installed collections instead of trying to fetch and parse them itself.

Given the main goal of antisibull-docs collection is to build documentation for multiple collections at once, it defaults to place the generated files into <dest-dir>/collections/<namespace>/<collection>. However, we only build documentation for one collection and thus pass --squash-hierarchy to avoid this longish path and make it generate documentation directly in <dest-dir>. Thanks to Felix for implementing this feature for us!

And that's it! We can generate our documentation with a single line now!

antsibull-docs collection --use-current --squash-hierarchy --dest-dir ./build/plugin_docs theforeman.foreman

The PR to switch to antsibull is open for review and I hope to get merged in soon!

Oh and you know what's cool? The documentation is now also available as a preview on ansible.com!


  1. Yes, the paths version of that setting is deprecated in 2.10, but as we support older Ansible versions, we still use it. 

24 July, 2020 08:01AM by evgeni

hackergotchi for Martin Michlmayr

Martin Michlmayr

beancount2ledger 1.1 released

Martin Blais recently announced that he'd like to re-organize the beancount code and split out some functionality into separate projects, including the beancount to ledger/hledger conversion code previously provided by bean-report.

I agreed to take on the maintenance of this code and I've now released beancount2ledger, a beancount to ledger/hledger converter.

You can install beancount2ledger with pip:

pip3 install beancount2ledger

Please report issues to the GitHub tracker.

There are a number of outstanding issues I'll fix soon, but please report any other issues you encounter.

Note that I'm not very familiar with hledger. I intend to sync up with hledger author Simon Michael soon, but please file an issue if you notice any problems with the hledger conversion.

Version 1.1 contains a number of fixes compared to the latest code in bean-report:

1.1 (2020-07-24)

  • Preserve metadata information (issue #3)
  • Preserve cost information (lot dates and lot labels/notes) (issue #5)
  • Avoid adding two prices in hledger (issue #2)
  • Avoid trailing whitespace in account open declarations (issue #6)
  • Fix indentation issue in postings (issue #8)
  • Fix indentation issue in price entries
  • Drop time information from price (P) entries
  • Add documentation
  • Relicense under GPL-2.0-or-later (issue #1)

1.0 (2020-07-22)

  • Split ledger and hledger conversion from bean-report into a standalone tool
  • Add man page for beancount2ledger(1)

24 July, 2020 07:04AM by Martin Michlmayr

July 23, 2020

hackergotchi for Sean Whitton

Sean Whitton

keyboardingupdates

Marks and mark rings in GNU Emacs

I recently attempted to answer the question of whether experienced Emacs users should consider partially or fully disabling Transient Mark mode, which is (and should be) the default in modern GNU Emacs.

That blog post was meant to be as information-dense as I could make it, but now I’d like to describe the experience I have been having after switching to my custom pseudo-Transient Mark mode, which is labelled “mitigation #2” in my older post.

In summary: I feel like I’ve uncovered a whole editing paradigm lying just beneath the surface of the editor I’ve already been using for years. That is cool and enjoyable in itself, but I think it’s also helped me understand other design decisions about the basics of the Emacs UI better than before – in particular, the ideas behind how Emacs chooses where to display buffers, which were very frustrating to me in the past. I am now regularly using relatively obscure commands like C-x 4 C-o. I see it! It all makes sense now!

I would encourage everyone who has never used Emacs without Transient Mark mode to try turning it off for a while, either fully or partially, just to see what you can learn. It’s fascinating how it can come to seem more convenient and natural to pop the mark just to go back to the end of the current line after fixing up something earlier in the line, even though doing so requires pressing two modified keys instead of just C-e.

Eshell

I was amused to learn some years ago that someone was trying to make Emacs work as an X11 window manager. I was amazed and impressed to learn, more recently, that the project is still going and a fair number of people are using it. Kudos! I suspect that the basic motivation for such projects is that Emacs is a virtual Lisp machine, and it has a certain way of managing visible windows, and people would like to be able to bring both of those to their X11 window management.

However, I am beginning to suspect that the intrinsic properties of Emacs buffers are tightly connected to the ways in which Emacs manages visible windows, and the intrinsic properties of Emacs buffers are at least as fundamental as its status as a virtual Lisp machine. Thus I am not convinced by the idea of trying to use Emacs’ ways of handling visible windows to handle windows which do not contain Emacs buffers. (but it’s certainly nice to learn it’s working out for others)

The more general point is this. Emacs buffers are as fundamental to Emacs as anything else is, so it seems unlikely to be particularly fruitful to move something typically done outside of Emacs into Emacs, unless that activity fits naturally into an Emacs buffer or buffers. Being suited to run on a virtual Lisp machine is not enough.

What could be more suited to an Emacs buffer, however, than a typical Unix command shell session? By this I mean things like running commands which produce text output, and piping this output between commands and into and out of files. Typically the commands one enters are sort of like tiny programs in themselves, even if there are no pipes involved, because you have to spend time determining just what options to pass to achieve what you want. It is great to have all your input and output available as ordinary buffer text, navigable just like all your other Emacs buffers.

Full screen text user interfaces, like top(1), are not the sort of thing I have in mind here. These are suited to terminal emulators, and an Emacs buffer makes a poor terminal emulator – what you end up with is a sort of terminal emulator emulator. Emacs buffers and terminal emulators are just different things.

These sorts of thoughts lead one to Eshell, the Emacs Shell. Quoting from its documentation:

The shell’s role is to make [system] functionality accessible to the user in an unformed state. Very roughly, it associates kernel functionality with textual commands, allowing the user to interact with the operating system via linguistic constructs. Process invocation is perhaps the most significant form this takes, using the kernel’s fork' andexec’ functions.

Emacs is … a user application, but it does make the functionality of the kernel accessible through an interpreted language – namely, Lisp. For that reason, there is little preventing Emacs from serving the same role as a modern shell. It too can manipulate the kernel in an unpredetermined way to cause system changes. All it’s missing is the shell-ish linguistic model.

Eshell has been working very well for me for the past month or so, for, at least, Debian packaging work, which is very command shell-oriented (think tools like dch(1)).

The other respects in which Eshell is tightly integrated with the rest of Emacs are icing on the cake. In particular, Eshell can transparently operate on remote hosts, using TRAMP. So when I need to execute commands on Debian’s ftp-master server to process package removal requests, I just cd /ssh:fasolo: in Eshell. Emacs takes care of disconnecting and connecting to the server when needed – there is no need to maintain a fragile SSH connection and a shell process (or anything else) running on the remote end.

Or I can cd /ssh:athena\|sudo:root@athena: to run commands as root on the webserver hosting this blog, and, again, the text of the session survives on my laptop, and may be continued at my leisure, no matter whether athena reboots, or I shut my laptop and open it up again the next morning. And of course you can easily edit files on the remote host.

23 July, 2020 06:58PM

spacecadetrebindings

I’ve been less good at taking adequate typing breaks during the lockdown and I’ve become concerned about how much chording my left hand does on its own during typical Emacs usage, with caps lock rebound to control, as I’ve had it for years.

I thought that now was as good a time as any to do something drastic about this. Here are my rebindings:

  • the keys on either side of the spacebar are control
  • the keys just outside of those are alt/meta
  • caps lock is Super, Windows or Command depending on OS
  • move any window manager keybindings which now become one handed left hand chords such that they are not.

Optional extras:

  • left control is caps lock
  • right control is the compose key.

This has the following advantages:

  • you can easily achieve this rebinding on GNU/Linux, Windows and macOS
  • almost every keyboard has enough keys near the spacebar to make it work, and it’s fine to have just one super key since it is not involved in any one handed chords
  • does not involve relying on the difference between tapping and releasing and holding a modifier key, which I find fragile
  • there are control and alt/meta keys available to both hands, so there is much less call for one-handed chording
  • control and alt/meta are pressed by the thumb, the strongest finger, so when one-handed chording does come up (e.g. C-x C-o without having to switch between control keys) it’s the least harmful form
    • my plan is to use the control/meta key available to the opposite hand for the first key of each sequence, and allow some one handed chording to complete the sequence.
      • Update 23/Jul/2020: I’ve found that I’m typing some of these sequences by performing a one-handed chord in order to type the second key, as planned, and others by performing a one-handed chord in order to type the first key. There are actually only a handful of these sequences, so it seems okay to have sequence-specific habits.
    • there is some temptation to use a curled up little finger on the new alt/meta key, I’m finding, but I’m trying to stop myself from doing that.

The main disadvantage, aside from an adjustment period when I feel that someone has inserted a massive marshmellow between me and my computer, is that Ctrl-Alt combinations are a bit difficult; in Emacs, C-M-SPC is hard to do without. However I think I’ve found a decent way to do it (thumb on control, curled ring finger on alt, possibly little finger on shift for Emacs’ infamous C-M-S-v standard binding). (Update 23/Jul/2020: this has been working out fine.)

23 July, 2020 04:47PM

Kinesis Advantage 2 for heavy Emacs users

A little under two months ago I invested in an expensive ergonomic keyboard, a Kinesis Advantage 2, and set about figuring out how to use it most effectively with Emacs. The default layout for the keyboard is great for strong typists who control their computer mostly with their mouse, but less good for Emacs users, who are strong typists that control their computer mostly with their keyboard.

It took me several tries to figure out where to put the ctrl, alt, backspace, delete, return and spacebar keys, and aside from one forum post I ran into, I haven’t found anyone online who came up with anything much like what I’ve come up with, so I thought I should probably write up a blog post.

The mappings

  • The pairs of arrow keys under the first two fingers of each hand become ctrl and alt/meta keys. This way there is a ctrl and alt/meta key for each hand, to reduce the need for one-handed chording.

    I bought the keyboard expecting to have all modifier keys on my thumbs. However, (i) only the two large thumb keys can be pressed without lifting your hand away from the home row, or stretching in a way that’s not healthy; and (ii) only the outermost large thumb key can be comfortably held down as a modifier.

    It takes a little work to get used to using the third and fifth fingers of one hand to hold down both alt/meta and shift, for typing core Emacs commands like M-^ and M-@, but it does become natural to do so.

  • The arrow keys are moved to the four ctrl/alt/super keys which run along the top of the thumb key areas.

  • The outermost large thumb key of each hand becomes a spacebar. This means it is easy to type C-u C-SPC with the right hand while the left hand holds down control, and sequences like C-x C-SPC and C-a C-SPC C-e with the left hand with the right hand holding down control.

    It took me a while to realise that it is not wasteful to have two spacebars.

  • The inner large thumb keys become backspace and return.

  • The international key becomes delete.

    Rarely needed for Emacs users, as we have C-d, so initially I just had no delete key, but soon came to regret this when trying to edit text in web forms.

  • Caps Lock becomes Super, but remains caps lock on the keypad layer.

    See my rebindings for ordinary keyboards for some discussion of having just a single Super key.

Sequences of two modified keys on different halves of the keyboard

It is desirable to input sequences like C-x C-o without switching which hand is holding the control key. This requires one-handed chording, but this is trecherous when the modifier keys not under the thumbs, because you might need to press the modified key with the same finger that’s holding the modifier!

Fortunately, most or all sequences of two keys modified by ctrl or alt/meta, where each of the two modifier keys is typed by a different hand, begin with C-c, C-x or M-g, and the left hand can handle each of these on its own. This leaves the right hand completely free to hit the second modified key while the left hand continues to hold down the modifier.

My rebindings for ordinary keyboards

I have some rebindings to make Emacs usage more ergonomic on an ordinary keyboard. So far, my Kinesis Advantage setup is close enough to that setup that I’m not having difficulty switching back and forth from my laptop keyboard.

The main difference is for sequences of two modified keys on different halves of the keyboard – which of the two modified keys is easiest to type as a one-handed chord is different on the Kinesis Advantage than on my laptop keyboard. At this point, I’m executing these sequences without any special thought, and they’re rare enough that I don’t think I need to try to determine what would be the most ergonomic way to handle them.

23 July, 2020 04:44PM

Dima Kogan

Finding long runs of "notable" data in a log

Here's yet another instance where the data processing I needed done could be acomplished entirely in the shell, with vnlog tools.

I have some time-series data in a text table. Via some join and filter operations, I have boiled down this table to a sequence of time indices where something interesting happened. For instance let's say it looks like this:

t.vnl

# time
1976
1977
1978
1979
1980
1986
1987
1988
1989
2011
2012
2013
2014
2015
4679
4680
4681
4682
4683
4684
4685
4686
4687
7281
7282
7283
7291
7292
7293

I'd like to find the longest contiguous chunk of time where the interesting thing kept happening. How? Like this!

$ < t.vnl vnl-filter -p 'time,d=diff(time)' |
          vnl-uniq -c -f -1 |
          vnl-filter 'd==1' -p 'count=count+1,time=time-1' |
          vnl-sort -nrk count |
          vnl-align
# count time
9       4679
5       2011
5       1976
4       1986
3       7291
3       7281

Bam! So the longest run was 9-frames-long, starting at time = 4679.

23 July, 2020 02:19PM by Dima Kogan

hackergotchi for Rapha&#235;l Hertzog

Raphaël Hertzog

Freexian’s report about Debian Long Term Support, June 2020

A Debian LTS logo Like each month, here comes a report about the work of paid contributors to Debian LTS.

Individual reports

In June, 202.00 work hours have been dispatched among 12 paid contributors. Their reports are available:

Evolution of the situation

June was the last month of Jessie LTS which ended on 2020-06-20. If you still need to run Jessie somewhere, please read the post about keeping Debian 8 Jessie alive for longer than 5 years.
So, as (Jessie) LTS is dead, long live the new LTS, Stretch LTS! Stretch has received its last point release, so regular LTS operations can now continue.
Accompanying this, for the first time, we have prepared a small survey about our users and contributors, who they are and why they are using LTS. Filling out the survey should take less than 10 minutes. We would really appreciate if you could participate in the survey online! On July 27th 2020 we will close the survey, so please don’t hesitate and participate now! After that, there will be a followup with the results.

The security tracker for Stretch LTS currently lists 29 packages with a known CVE and the dla-needed.txt file has 44 packages needing an update in Stretch LTS.

Thanks to our sponsors

New sponsors are in bold.

We welcome CoreFiling this month!

No comment | Liked this article? Click here. | My blog is Flattr-enabled.

23 July, 2020 02:10PM by Raphaël Hertzog

July 22, 2020

hackergotchi for Bits from Debian

Bits from Debian

Let's celebrate DebianDay 2020 around the world

We encourage our community to celebrate around the world the 27th Debian anniversary with organized DebianDay events. This year due to the COVID-19 pandemic we cannot organize in-person events, so we ask instead that contributors, developers, teams, groups, maintainers, and users promote The Debian Project and Debian activities online on August 16th (and/or 15th).

Communities can organize a full schedule of online activities throughout the day. These activities can include talks, workshops, active participation with contributions such as translations assistance or editing, debates, BoFs, and all of this in your local language using tools such as Jitsi for capturing audio and video from presenters for later streaming to YouTube.

If you are not aware of any local community organizing a full event or you don't want to join one, you can solo design your own activity using OBS and stream it to YouTube. You can watch an OBS tutorial here.

Don't forget to record your activity as it will be a nice idea to upload it to Peertube later.

Please add your event/activity on the DebianDay wiki page and let us know about and advertise it on Debian micronews. To share it, you have several options:

  • Follow the steps listed here for Debian Developers.
  • Contact us using IRC in channel #debian-publicity on the OFTC network, and ask us there.
  • Send a mail to debian-publicity@lists.debian.org and ask for your item to be included in micronews. This is a publicly archived list.

PS: DebConf20 online is coming! It will be held from August 23rd to 29th, 2020. Registration is already open.

22 July, 2020 01:30PM by Paulo Henrique de Lima Santana (phls)

July 21, 2020

hackergotchi for Jonathan Dowland

Jonathan Dowland

FlashFloppy OLED display

This is the tenth part in a series of blog posts. The previous post was Amiga floppy recovery project: what next?. The whole series is available here: Amiga.

Rotary encoder, OLED display and mount

Rotary encoder, OLED display and mount

I haven't made any substantive progress on my Amiga floppy recovery project for a while, but I felt like some retail therapy a few days ago so I bought a rotary encoder and OLED display for the Gotek floppy disk emulator along with a 3D-printed mount for them. I'm pleased with the results! The rather undescriptive "DSKA0001" in the picture is a result of my floppy image naming scheme: the display is capable of much more useful labels such as "Lemmings", "Deluxe Paint IV", etc.

The Gotek and all the new bits can now be moved inside the Amiga A500's chassis.

21 July, 2020 02:34PM

hackergotchi for Bits from Debian

Bits from Debian

New Debian Developers and Maintainers (May and June 2020)

The following contributors got their Debian Developer accounts in the last two months:

  • Richard Laager (rlaager)
  • Thiago Andrade Marques (andrade)
  • Vincent Prat (vivi)
  • Michael Robin Crusoe (crusoe)
  • Jordan Justen (jljusten)
  • Anuradha Weeraman (anuradha)
  • Bernelle Verster (indiebio)
  • Gabriel F. T. Gomes (gabriel)
  • Kurt Kremitzki (kkremitzki)
  • Nicolas Mora (babelouest)
  • Birger Schacht (birger)
  • Sudip Mukherjee (sudip)

The following contributors were added as Debian Maintainers in the last two months:

  • Marco Trevisan
  • Dennis Braun
  • Stephane Neveu
  • Seunghun Han
  • Alexander Johan Georg Kjäll
  • Friedrich Beckmann
  • Diego M. Rodriguez
  • Nilesh Patra
  • Hiroshi Yokota

Congratulations!

21 July, 2020 02:00PM by Jean-Pierre Giraud

July 20, 2020

hackergotchi for Evgeni Golov

Evgeni Golov

Building and publishing documentation for Ansible Collections

I had a draft of this article for about two months, but never really managed to polish and finalize it, partially due to some nasty hacks needed down the road. Thankfully, one of my wishes was heard and I had now the chance to revisit the post and try a few things out. Sadly, my wish was granted only partially and the result is still not beautiful, but read yourself ;-)

UPDATE: I've published a follow up post on building documentation for Ansible Collections using antsibull, as my wish was now fully granted.

As part of my day job, I am maintaining the Foreman Ansible Modules - a collection of modules to interact with Foreman and its plugins (most notably Katello). We've been maintaining this collection (as in set of modules) since 2017, so much longer than collections (as in Ansible Collections) existed, but the introduction of Ansible Collections allowed us to provide a much easier and supported way to distribute the modules to our users.

Now users usually want two things: features and documentation. Features are easy, we already have plenty of them. But documentation was a bit cumbersome: we had documentation inside the modules, so you could read it via ansible-doc on the command line if you had the collection installed, but we wanted to provide online readable and versioned documentation too - something the users are used to from the official Ansible documentation.

Building HTML from Ansible modules

Ansible modules contain documentation in form of YAML blocks documenting the parameters, examples and return values of the module. The Ansible documentation site is built using Sphinx from reStructuredText. As the modules don't contain reStructuredText, Ansible hashad a tool to generate it from the documentation YAML: build-ansible.py document-plugins. The tool and the accompanying libraries are not part of the Ansible distribution - they just live in the hacking directory. To run them we need a git checkout of Ansible and source hacking/env-setup to set PYTHONPATH and a few other variables correctly for Ansible to run directly from that checkout.

It would be nice if that'd be a feature of ansible-doc, but while it isn't, we need to have a full Ansible git checkout to be able to continue.The tool has been recently split out into an own repository/distribution: antsibull. However it was also a bit redesigned to be easier to use (good!), and my hack to abuse it to build documentation for out-of-tree modules doesn't work anymore (bad!). There is an issue open for collections support, so I hope to be able to switch to antsibull soon.

Anyways, back to the original hack.

As we're using documentation fragments, we need to tell the tool to look for these, because otherwise we'd get errors about not found fragments. We're passing ANSIBLE_COLLECTIONS_PATHS so that the tool can find the correct, namespaced documentation fragments there. We also need to provide --module-dir pointing at the actual modules we want to build documentation for.

ANSIBLEGIT=/path/to/ansible.git
source ${ANSIBLEGIT}/hacking/env-setup
ANSIBLE_COLLECTIONS_PATHS=../build/collections python3 ${ANSIBLEGIT}/hacking/build-ansible.py document-plugins --module-dir ../plugins/modules --template-dir ./_templates --template-dir ${ANSIBLEGIT}/docs/templates --type rst --output-dir ./modules/

Ideally, when antsibull supports collections, this will become antsibull-docs collection … without any need to have an Ansible checkout, sourcing env-setup or pass tons of paths.

Until then we have a Makefile that clones Ansible, runs the above command and then calls Sphinx (which provides a nice Makefile for building) to generate HTML from the reStructuredText.

You can find our slightly modified templates and themes in our git repository in the docs directory.

Publishing HTML documentation for Ansible Modules

Now that we have a way to build the documentation, let's also automate publishing, because nothing is worse than out-of-date documentation!

We're using GitHub and GitHub Actions for that, but you can achieve the same with GitLab, TravisCI or Jenkins.

First, we need a trigger. As we want always up-to-date documentation for the main branch where all the development happens and also documentation for all stable releases that are tagged (we use vX.Y.Z for the tags), we can do something like this:

on:
  push:
    tags:
      - v[0-9]+.[0-9]+.[0-9]+
    branches:
      - master

Now that we have a trigger, we define the job steps that get executed:

    steps:
      - name: Check out the code
        uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: "3.7"
      - name: Install dependencies
        run: make doc-setup
      - name: Build docs
        run: make doc

At this point we will have the docs built by make doc in the docs/_build/html directory, but not published anywhere yet.

As we're using GitHub anyways, we can also use GitHub Pages to host the result.

      - uses: actions/checkout@v2
      - name: configure git
        run: |
          git config user.name "${GITHUB_ACTOR}"
          git config user.email "${GITHUB_ACTOR}@bots.github.com"
          git fetch --no-tags --prune --depth=1 origin +refs/heads/*:refs/remotes/origin/*
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: "3.7"
      - name: Install dependencies
        run: make doc-setup
      - name: Build docs
        run: make doc
      - name: commit docs
        run: |
          git checkout gh-pages
          rm -rf $(basename ${GITHUB_REF})
          mv docs/_build/html $(basename ${GITHUB_REF})
          dirname */index.html | sort --version-sort | xargs -I@@ -n1 echo '<div><a href="@@/"><p>@@</p></a></div>' >> index.html
          git add $(basename ${GITHUB_REF}) index.html
          git commit -m "update docs for $(basename ${GITHUB_REF})" || true
      - name: push docs
        run: git push origin gh-pages

As this is not exactly self explanatory:

  1. Configure git to have a proper author name and email, as otherwise you get ugly history and maybe even failing commits
  2. Fetch all branch names, as the checkout action by default doesn't do this.
  3. Setup Python, Sphinx, Ansible etc.
  4. Build the documentation as described above.
  5. Switch to the gh-pages branch from the commit that triggered the workflow.
  6. Remove any existing documentation for this tag/branch ($GITHUB_REF contains the name which triggered the workflow) if it exists already.
  7. Move the previously built documentation from the Sphinx output directory to a directory named after the current target.
  8. Generate a simple index of all available documentation versions.
  9. Commit all changes, but don't fail if there is nothing to commit.
  10. Push to the gh-pages branch which will trigger a GitHub Pages deployment.

Pretty sure this won't win any beauty contest for scripting and automation, but it gets the job done and nobody on the team has to remember to update the documentation anymore.

You can see the results on theforeman.org or directly on GitHub.

20 July, 2020 07:17PM by evgeni

hackergotchi for Steinar H. Gunderson

Steinar H. Gunderson

Reverse-engineering the FIRST marathon program

Last year, I ran my first marathon ever (at 3:07:52, in the fairly hilly Oslo course), using the FIRST marathon program (which, despite the name, is not necessarily meant for beginners). This year, as the Covid-19 lockdowns started, I decided to go for it again using the same program, but there was one annoyance; I wanted to change target times as it became obvious my initial target got too easy, but there's no way to calculate it electronically.

FIRST comes in the form of a book; you can find an older version of the 10K and marathon programs if you search a bit online, but fundamentally, the way it works is that you declare a 5K personal best (whether true or not), look up a bunch of tempos in a table in the book from that, and then use that to get three runs every week. (You also do cross-training and strength training, or at least that's the idea.) For instance, the book might say that this week's track intervals are 6x 800m, so you go look up your 800m interval times in the table. If you have a 5K PB of 19:30, the book might say that 800m interval times are 2:52 (3:35/km), so off you go running.

The tables are never explained, and they don't match up with the formulas that were published in the earlier versions. There's at least one running calculator that can derive FIRST paces, but it defaults to miles and has a different calculation for marathon pace (which sometimes creates absurd situations like “long tempo” being slower than “marathon pace”), so I really wanted to just get the formulas to input into my own spreadsheets.

Enter regression. I just typed in a bunch of the tables, graphed them, saw that everything was on a dead straight line (R=1.00 for linear regression) and got the constants from there. So without further ado:

If you can run 5K at x seconds per kilometer, the Holy Gospel of FIRST declares that you can run 42.195K at 1.15313x seconds. (I am sure there are more sophisticated models, but perhaps this is good enough?) Incidentally or not, this means an 18:30 5K becomes nearly exactly three hours on a marathon (only two seconds away). (I didn't bother with the 10K and half-marathon estimation paces; there are so many numbers to input).

The tempo run paces are even simpler. Take your 5K pace, and add 10 sec/km, and that's short tempo (ST). Medium tempo (MT) is 5K + 20 sec/km. Long tempo (LT) is 5K + 29 sec/km.

That leaves only the track repeats. For this, first take the 5K pace and multiply by 1.00579, leaving what I will call the “reference pace” (RP). I don't know if this constant carries any particular meaning, and obviously, it's nowhere in the book; it's just the slope of the regression. 400m time is 400m at RP, minus 10 seconds. (That is 10 seconds absolute time, not 10 seconds/km. So if you have an 18:30 5K PB, you'll have an 18:36 5K at RP, which is 1:29 400m at RP, which then gives a 1:19 400m.)

Similarly: 600m is -13 seconds, 800m is -16 seconds, 1000m is -18 seconds, 1200m is also -18 seconds, 1600m is -16 seconds, and 2000m (which is specified, but seemingly never used in any of the programs) is -15 seconds. You can see two effects going against each other here; longer intervals mean more seconds to shave off for a given pace, but they also give lower pace, and thus the U-like shape.

And that's all there is to it. Happy running, and may there be a good race close to you!

20 July, 2020 07:05PM

Dominique Dumont

Security gotcha with log collection on Azure Kubernetes cluster.

Azure Kubernetes Service provides a nice way to set up Kubernetes
cluster in the cloud. It’s quite practical as AKS is setup by default
with a rich monitoring and reporting environment. By default, all
container logs are collected, CPU and disk data are gathered. �

I used AKS to setup a cluster for my first client as a
freelance. Everything was nice until my client asked me why logs
collection was as expensive as the computer resources.💸

Ouch… 🤦

My first reflex was to reduce the amount of logs produced by all our
containers, i.e. start logging at warn level instead of info
level
. This reduced the amount of logs quite a lot.

But this did not reduce the cost of collecting logs, which looks like
to a be a common issue.

Thanks to the documentation provided by Microsoft, I was able to find
that ContainerInventory data table was responsible of more than 60%
of our logging costs.

What is ContainerInventory ? It’s a facility to monitor the content
of all environment variables from all containers.

Wait… What ? ⚠

Should we be worried about our database credentials which are, legacy
oblige, stored in environment variables ?

Unfortunately, the query shown below confirmed that, yes, we should:
the logs aggregated by Azure contains the database credentials of my
client.

ContainerInventory
| where TimeGenerated > ago(1h)

Having credentials collected in logs is lackluster from a security
point of view. 🙄

And we don’t need it because our environment variables do not change.

Well, it’s now time to fix these issues. 🛠

We’re going to:

  1. disable the collection of environment variables in Azure, which
    will reduce cost and plug the potential credential leak
  2. renew all DB credentials, because the previous credentials can be
    considered as compromised (The renewal of our DB passwords is quite
    easy with the script I provided to my client)
  3. pass credentials with files instead of environment variables.

In summary, the service provided by Azure is still nice, but beware of
the default configuration which may contain surprises.

I’m a freelance, available for hire. The https://code-straight.fr site
describes how I can help your projects.

All the best

 

20 July, 2020 04:26PM by dod