February 28, 2021

Dima Kogan

mrcal: principled camera calibrations

This is a big deal.

In my day job I work with images captured by cameras, using those images to infer something about the geometry of the scene being observed. Naturally, to get good results you need to have a good estimate of the behavior of the lens (the "intrinsics"), and of the relative geometry of the cameras (the "extrinsics"; if there's more than one camera).

The usual way to do this is to perform a "calibration" procedure to compute the intrinsics and extrinsics, and then to use the resulting "camera model" to process the subsequent images. Wikipedia has an article. And from experience, the most common current toolkit to do this appears to be OpenCV.

People have been doing this for a while, but for whatever reason the existing tools all suck. They make basic questions like "how much data should I gather for a calibration?" and "how good is this calibration I just computed?" and "how different are these two models?" unanswerable.

This is clearly seen from the links above. The wikipedia article talks about fitting a pinhole model to lenses, even though no real lenses follow this model (telephoto lenses do somewhat; wider lenses don't at all).

And the OpenCV tutorial cheerfully says that

Re-projection error gives a good estimation of just how exact the found
parameters are. The closer the re-projection error is to zero, the more accurate
the parameters we found are.

This statement is trivially proven false: throw away most of your calibration data, and your reprojection error becomes very low. But we can all agree that a calibration computed from less data is actually worse. Right?

All the various assumptions and hacks in the existing tooling are fine as long as you don't need a whole lot of accuracy out of your results. I need a lot of accuracy, however, so all the existing tools don't work for my applications.

So I built a new set of tools, and have been using them with great delight. I just got the blessing to do a public release, so I'm announcing it here. The tools are

  • mrgingham: a chessboard corner finder. OpenCV has one, but as far as I can tell, it doesn't work; and it is very slow to tell you that. mrgingham is relatively quick, robust to all sorts of lens behaviors, and reports the accuracy of its output. This is a C++ library with Python bindings, and a commandline tool. 99% of the time the commandline tool is what I use.
  • mrcal: a large toolkit to run calibrations, to manipulate images and camera models in all sorts of ways, and to visualize stuff. This toolkit does a lot. It's a C library and a Python library and a number of commandline tools. Currently the C library exists primarily in the service of the other two, but it's already very capable, and will become more so over time.

mrcal does a whole lot to produce calibrations that are as good as possible, and it will tell you just how good they are, and it includes visualization capabilities for extensive user feedback. An overview of the capabilities of the toolkit (with lots of pretty pictures!) is at the tour of mrcal.

There's a lot of documentation and examples, but up to now I have been the primary user of the tools. So I expect this to be somewhat rough when others look at it. Bug reports and patches are welcome.

mrcal is an excellent base, but it's nowhere near "done". The documentation has some notes about the planned features and improvements, and I'm always reachable by email.

Let me know if you try it out!

28 February, 2021 08:12PM by Dima Kogan

hackergotchi for Daniel Pocock

Daniel Pocock

Gangstalking and victim-blaming

I will destroy you, threat, Washington

It is ironic that the first person to depart the Biden administration was sanctioned for threatening somebody else's career.

This week Marko Rodriguez went public with news that rogue members of the Apache Software Foundation had decided to persecute him for his commentary on social issues. The board had voted to reclassify satire as a form of prose that "borders" on hate speech. Either it is hate speech or it isn't. To suggest it "borders" on hate speech is a fudge. The sly comparison of these very different types of writing is simply a smear to hurt his career.

To put this in perspective, board members who disagreed with this defamation did not only vote against it but also choose to resign.

Around Valentine's Day, Brittany Higgins, a former employee of Australia's Minister for Defence went public with news about being raped on the ministerial sofa. The questions this woman raises are extraordinary, for example, if the Minister for Defence, Linda Reynolds, can not defend the new girl in the office, how can we rely on her to defend our country?

Brittany Higgins, Australia

Higgins chose not to name the accused publicly. It appears she wishes to focus attention on the culture and the cover-up. Two independent news organizations, True Crimes News Weekly and independent journalist Shane Olsen have identified a suspect. There is now a twitter hashtag too. A Youtube video shows the former Attorney General, George Brandis, praising Bruce Lehrmann and other former staff in the presence of high court justices.

George Brandis (former Attorney General): All of us know how important staff are to us. We spend so much time together, mostly away from home. We share so many experiences that they become like a second family.

As the man departed days after the incident in 2019, it appears that the Government have had plenty of time to remove his name from virtually all official web sites although there is no super-injunction (yet) to prevent discussion of his identity.

Against this backdrop, Google admitted two female researchers subject to high-profile sackings may have been doing legitimate research. Like Rodriguez and Higgins, both of Google's female victims had been threatened to self-censor, they refused, they were shamed, they bravely chose to put their persecution in the public domain.

All these cases inevitably remind me of other cases, the growing body count in the free and open source software world.

Higgins' decision to go public helps us all see how a cover-up was built from day one. Her boss, Linda Reynolds, had suggested that pursuing a criminal justice complaint would destroy Miss Higgins' career. In effect, the victim was blackmailed to stay silent. This is the thread that draws all these cases of oppression together. In December 2018, two long standing volunteers in the free, open source software world, Dr Norbert Preining and I, revealed how we were subject to blackmail and coercion in our respective roles. In our cases, we both received the veiled threats in writing:

Knife at throat, Debian Account Managers, DAM, blackmail
We are sending this email privately, leaving its disclosure as your decision (although traces in public databases are unavoidable)

In other words, they are saying that if we call out the coercive nature of their communications, they will seek to destroy us.

When you receive a threat like this from somebody with a history of publicly shaming people on a hideous scale, it really feels like they are holding a knife to your throat.


In my case, the community of volunteers and donors had clearly elected me as the fellowship representative so this blackmail was an attack on all those who voted. It was my duty to inform people and call it out.

The crimes were very different but the message seems to be the same: the organization must be protected at any cost. When those in authority do something wrong, the victims have to stay silent, grin and bear it or some gang will impose a bigger pain on the victim.

More on the former Debian Project Leader (DPL), Chris Lamb, giving negative references for volunteers

One volunteer sent me the following comments about Chris Lamb. Many people receiving copies of defamation have showed it to the survivors:

Volunteer: But I am scared that Lamb actually also hosed an application for a company in NY, a job related to Debian. If that has happened, and I can reasonably document it, I would consider a defamation law suit

When the leader of any organization, whether it is Apache, Debian or Google, uses the authority of their position to push defamation, it is like using the height of a bridge to stand above a freeway and drop bricks onto the cars underneath. Lamb may not fear consequences for his actions, his father is a barrister, Robert Lamb, who appears well qualified to stifle any volunteers seeking redress.

28 February, 2021 07:00PM

hackergotchi for Chris Lamb

Chris Lamb

Free software activities in February 2021

Here is my monthly update covering what I have been doing in the free software world during February 2021 (previous month):

  • Reviewed and merged a number of contribution from Peter Law to my django-cache-toolbox library for Django-based web applications, including: support always fetching some relations when loading a model (#27), allow use of custom auth.User model. (#29), avoid some more database calls (#30), wrap some collections in tuples for compatibility (#32) and cope with only some of our related models actually being loaded (#33).
  • As part of my role of being the assistant Secretary of the Open Source Initiative and a board director of Software in the Public Interest, I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet, as well as the usual internal discussions, etc.

  • Opened a pull request to fix the relative target of manpage links in Roger Wesson's mocassin library. [...]


Reproducible Builds

The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during compilation process by promising identical results are always generated from a given source, therefore allowing multiple third-parties to come to a consensus on whether a build was compromised.

The project is proud to be a member project of the Software Freedom Conservancy. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.

This month, I:

I also made the following changes to diffoscope, including preparing and uploading versions 167 and 168 to Debian:

  • Bug fixes:

    • Don't call difflib.Differ.compare with very large inputs; it is at least O(n^2) and makes diffoscope (appear to) hang. [...]
    • Don't rely on dumpimage returning an appropriate exit code; check that the file actually exists. [...]
    • Don't rely on magic.Magic to have an identical API between file's magic.py and PyPI's python-magic library. [...]
  • Revamp temporary file handling:

    • Ensure we cleanup our temporary directory by avoiding confusion between the TemporaryDirectory instance and underlying directory. (#981123)
    • Try and use a potentially-useful suffix to our temporary directory. [...]
  • Testsuite improvements:

    • Strip newlines when determining Black version to avoid requires black >= 20.8b1 (18.9b0\n detected) in test output. [...]
    • Fix weakref-related handling in Python 3.7 (i.e. Debian buster). [...]
    • If our temporary directory does not exist anymore, recreate it. [...]
    • Fix FIT-related tests in Debian buster [...] and fit_expected_diff [...].
    • Gnumeric is back in testing so re-add to (test) Build-Depends. [...]
    • Mark test_apk.py::test_android_manifest as being allowed to fail for now. [...]
    • Add u-boot-tools to (test) Build-Depends so salsa.debian.org pipelines test the new U-Boot FIT comparator. [...]
    • Move to assert_diff utility in a number of tests. [...][...]
  • Codebase improvements:

    • Correct capitalisation of 'jQuery'. [...]
    • Update my copyright years. [...]
    • Tidy imports in diffoscope.comparators.fit. [...]
    • Don't use Inheriting PATH of X, use PATH is X in logging messages. [...]
    • Drop unused Config.acl and Config.xattr attributes [...] and set a default Config.extended_filesystem_attributes. [...]




  • python-django:

  • redis:

    • 6.0.10-4 — New upstream release, fixing cluster access to unaligned memory on ARM architectures with hard alignment requirements (such as armhf and arm64). (#982504)
    • 6.0.11-1 — New upstream release, incorporating security fixes. (#983446)
    • 6.2~rc3-1 — New upstream release candidate.
    • 6.2.0-1 — New upstream stable release, incorporating security fixes. (#983446)
  • gunicorn (20.1.0-1) — New upstream release.

  • xtrlock (2.15) — Also support the XK_KP_Enter key (i.e. the Enter on the numeric keypad) to submit a password. (#982634)

I also sponsored an upload of adminer (4.7.9-1) for Alexandre Rossi.

Debian LTS

This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 hours on its sister Extended LTS project.

You can find out more about the project via the following video:

28 February, 2021 06:16PM

hackergotchi for Martin-Éric Racine

Martin-Éric Racine

Shipping Debian with GNOME X.XX.0 is an extremely bad idea

Since the freeze has slowly crept in, now is the time to revisit my pet peeve with Debian's release process: to publish a new Debian release as soon as GNOME published a new X.XX.0 version. This is an extremely bad idea: X.XX.0 releases tend to lack polish, their translations are not up-to-date and several silly bugs that hamper the user experience (what the Ubuntu guys call "paper cuts") exist. Those issues tend to be fixed later when GNOME X.XX.1, X.XX.2, etc. bugfix releases are published. However, Debian has a policy of not pushing non-security releases onto a stable distribution. In this particular case, there are only two valid alternatives: either release Bullseye with GNOME 3.38.X or change the Debian policy to allow pushing 3.40.X bugfix releases via bullseye-updates.

28 February, 2021 02:46PM by Martin-Éric (noreply@blogger.com)

February 27, 2021

Russ Allbery

DocKnot 4.01

DocKnot is my software documentation and release management tool. This release adds support for a global user configuration file separate from the metadata for any given project and adds support for signing generated distribution tarballs with GnuPG. Currently, the only configuration options for the global configuration file are to set the destination location of generated distributions and the PGP key to use when signing them.

This release also removes some now-unnecessary helper functions, fixes docknot --help, and cleans up some documentation bugs left over from the big changes in 4.00.

You can get the latest release from CPAN or from the DocKnot distribution page.

27 February, 2021 09:11PM

Russell Coker

Dima Kogan

horizonator: terrain renderer based on SRTM DEMs

Check this out:


I just resurrected and cleaned up an old tool I had lying around. It's now nice and usable by others. This tool loads terrain data, and renders it from the ground, simulating what a human or a camera would see. This is useful for armchair exploring or for identifying peaks. This was relatively novel when I wrote it >10 years ago, but there are a number of similar tools in existence now. This implementation is still useful in that it's freely licensed and contains APIs, so fancier processing can be performed on its output.

Sources and (barely-complete-enough) documentation live here:


27 February, 2021 12:14PM by Dima Kogan

February 26, 2021

hackergotchi for Ritesh Raj Sarraf

Ritesh Raj Sarraf

Wayland KDE X11

KDE Impressions

These days, I often hear a lot about Wayland. And how much of effort is being put into it; not just by the Embedded world but also the usual Desktop systems, namely KDE and GNOME.

In recent past, I switched back to KDE and have been (very) happy about the switch. Even though the KDE 4 (and initial KDE 5) debacle had burnt many, coming back to a usable KDE desktop is always a delight. It makes me feel home with the elegance, while at the same time the flexibility, it provides. It feels so nice to draft this blog article from Kwrite + VI Input Mode

Thanks to the great work of the Debian KDE Team, but Norbert Preining in particular, who has helped bring very up-to-date KDE packages into Debian. Right now, I’m on a Plamsa 5.21.1 desktop, which is recent by all standards.


Almost all the places in the Linux world these days are busy with integrating Wayland as the primary display service. Not sure what the current status on the GNOME side is but I definitely keep trying KDE + Wayland with every release.

I keep trying with every release because it still is not prime for daily use. And it makes me get back to X11, no matter how dated some may call. Fact is, X11 still shines to me as an end-user.

Glitches with Wayland still are (Based on this week’s test on Plasma 5.21.1):

  • Horrible performance compared to X11
  • Very crashy, especially when hotplugging secondary display. Plasma would just crash. X11 is very resilient to such things, part of the reason I can think is the age of the codebase.
  • Many many applications still need to be fixed for Wayland. Or Wayland needs to accomodate them in some way. XWayland does not really feel like the answer.

And while KDE keeps insisting users to switch to Wayland, as that’s where all the new enhancements and fixes are put in, someone like me still needs to stick to X11 for the time being. So to get my shiny new LG 27" 4K Monitor (3840x2160 60.00*+) to work without too much glitch, I had to live with an alias:

$ alias | grep xrandr
alias rrs_xrandr_lg='xrandr --output DP-1 --mode 3840x2160 --scale .75x.75'
18:31 ♒ � ♅ ♄ ⛢     ☺ 😄  

Plasma 5.21

On the brighter side, the Plasma 5.21.1 release brings some nice enhancements in other areas.

  • I’m now able to make use of tighter integration with systemd/cgroups, with better organization and management of processes overall.
  • The new Plasma theme, Breeze Twilight, is a good blend of Light + Dark.

I also appreciate the work put in by Michail Vourlakos. The KDE project is lucky to have a developer/designer like him. His vision and work into the KDE desktop is well beyond a writing by me.

$ usystemctl status plasma-plasmashell.service 
â—� plasma-plasmashell.service - KDE Plasma Workspace
     Loaded: loaded (/usr/lib/systemd/user/plasma-plasmashell.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-02-26 18:34:23 IST; 13s ago
   Main PID: 501806 (plasmashell)
      Tasks: 21 (limit: 18821)
     Memory: 759.8M
        CPU: 13.706s
     CGroup: /user.slice/user-1000.slice/user@1000.service/session.slice/plasma-plasmashell.service
             └─501806 /usr/bin/plasmashell --no-respawn
Feb 26 18:35:00 priyasi plasmashell[501806]: qml: recreating buttons
Feb 26 18:35:21 priyasi plasmashell[501806]: qml: recreating buttons
Feb 26 18:35:49 priyasi plasmashell[501806]: qml: recreating buttons
Feb 26 18:35:57 priyasi plasmashell[501806]: qml: recreating buttons
18:36 ♒ � ♅ ♄ ⛢     ☺ 😄

OBS - Open Build Service

I should also thank the OpenSUSE folks for the OBS work. It has enabled the close equivalent (or better, in my experience) of PPAs for Debian. And that is what has enabled developers like Norbert to easily and quickly be able to deliver the entire KDE suite.

26 February, 2021 11:38AM by Ritesh Raj Sarraf (rrs@researchut.com)

February 24, 2021

hackergotchi for Rogério Brito

Rogério Brito

Alternatives to ikiwiki?

It's been quite a long time since I last posted anything on this blog and I can say that one of the reasons for that I don't feel comfortable using ikiwiki anymore. ☹

I am actively looking for alternatives to ikiwiki that allow me, mainly, to write blog posts with the following characteristics (not necessarily in order of importance):

  • Write in a lightweight markup language (e.g., markdown), with some features, like creating tables, having footnotes etc. I may be open to using something else (like restructured text or, perhaps, asciidoc, about which I know next to nothing), if necessary.
  • Store what I do in a git repository, to be future-proof.
  • Create a static site from a version of what I wrote in the lightweight markup language, which is especially useful for hosting the site with, say, GitHub pages, until I find where I can host my site. (My old email and account of more than 25 years is being closed and I will be "homeless", which is unfortunate).
  • Use mathematics, extensively, mostly copying and pasting from LaTeX documents.

    This, is one major pain point with ikiwiki. Apparently almost nobody cares about supporting MathJax/KaTex out-of-the-box. ☹ To add insult to injury, the templates are very general to the point of being very hard to read (read: feature bloat) and it doesn't help that my editor (emacs) doesn't know (at least in its current configuration) how to display its structure.

    I have tried to use hugo (which we happily have packaged in Debian), but configuring it is totally crazy: first, you have to decide on a theme and, then, (almost?) everything that you do is tied to that theme. This is (almost) the opposite to the philosophy of LaTeX: first, write your text and, only then, worry about the style/looks. Separation of content and form doesn't seem to be the priority, from a few days looking a it.1

    I have more to say about hugo and I failed, but I would still like to give it a try, if I don't find anything else that has the features that I'm looking for.
  • Enable syntax-highlighting for code samples. I like to post my discoveries regarding programs and programming languages and, of course, having readable code in my blog would be a very good thing.2
  • Be packaged in Debian. We all love the convenience that this brings, of course.

Connected to the fact that I only can have static sites (no CGI, no forms, nothing else), I am, at this time, using Disqus to host the comments of my blog. I am also thinking of alternatives to this, like sending people to Twitter (or mastodon or email) or some site similar to Disqus, but with more of a Free Software inclination.

Anyway, I am almost ready for any kind of transition, since I have already converted most posts (of course, not yet this one ☺) with some Python scripts to a format that I feel is a bit more format-agnostic than what ikiwiki uses.

  1. That's not to mention the myriad of hugo themes and theme authors that try to bribe you into using their hosted solutions (despite branding everything as "open source! OMG!"), like "wowchemy"—you will have a hard time untangling the instructions of their so bloated themes to be usable on your local computer; so much so to the point that you give up with their convoluted configuration (which, potentially, doesn't "transfer" to other themes, if you are worried about possibly changing themes in the future). ↩

  2. I like the style of fenced blocks that GitHub used, where you prefix the code with the name of the language to give a hint of how to highlight the code snipped. ↩

24 February, 2021 09:38PM

February 23, 2021

hackergotchi for Daniel Pocock

Daniel Pocock

POWER9, ARM64 and 64k page sizes


I've recently had discussions with other developers in the Fedora world about the default 64k page size on POWER9. The vast majority of GNU/Linux users have a 4k page size. There is now a change proposal for the ppc64le page size on Fedora 35 and a related discussion on the devel mailing list.

Why and when the non-x86 architectures are relevant

With each new generation of x86 processors from Intel and AMD there is a larger quantity of opaque microcode that independent developers are unable to audit or fix.

When a vendor has such an incredible market share, it is inevitable that problems like this will arise.

Investing some time and effort on alternative architectures is good insurance: when the day comes that this microcode is compromised, some percentage of users will jump ship.

Getting started on POWER9 and ARM64 is easier than ever

For POWER9, please see my recent blog about the Talos II Quickstart.

Similar blogs have recently appeared about ARM64, for example, this Fedora Magazine article about SolidRun HoneyComb LX2K.

Choosing between POWER9 and ARM64

ARM64 may be a better choice if you are very sensitive to the heat emissions and energy consumption costs.

POWER9 may be a better choice if you need a lot of compute power.

While neither ARM64 nor POWER9 have microcode comparable to Intel or AMD products, it is still important to look at the overall system. For example, Raptor's Talos II motherboard has the FSF Respects Your Freedom (RYF) certification but if you order the optional SATA controller, you end up with some proprietary firmware.

Why 64k page sizes are an issue

The GNU/Linux kernel for these platforms can be compiled with either 4k or 64k page size. The distribution chooses which of these options to select. The kernel created by the distribution is included in the installation disk for the distribution.

One acute consequence of this is the relationship between Btrfs sectorsize and kernel page size. Btrfs filesystems can only be used on systems with the same page size. The Btrfs driver is being improved to remove this restriction but for users of Fedora 34 and older systems, this is a very inconvenient issue. If you need to move Btrfs filesystems between systems with different page sizes then they simply won't work.

It appears that nobody tests the kernel and amdgpu drivers on these non-standard page sizes before each official release. Consequently, if there is a problem, it is only discovered by users after the upstream release. This means that users on these platforms are always a step behind users on other platforms.

Improving support for 64k page sizes

Personally, I'm quite keen to see the 64k page size succeed.

I believe that is only possible when these platforms have critical mass and when some of the upstream developers use these machines on a daily basis.

Until we get to that point, I feel it is a chicken-and-egg problem: things don't work, so people buy in more slowly, so there are less people to report bugs and/or fix things.

Automated CI testing of the kernel and amdgpu code may also help to catch some issues before official releases.

To summarize, I have an open mind about how to go about this. Please feel free to share your ideas and experiences in the discussion or through your blog.


Anybody who buys one of these machines today can still use it almost immediately.

One option is to use a distribution with a 4k page size or compile your own kernel with a 4k page size.

Another idea is to simply avoid using Btrfs for another six months: if you use the installation system of your preferred distribution to create filesystems, check that it isn't using Btrfs. You may need to manually override it to use ext4 for the moment.

If you don't need a modern GPU, some of the previous generation, such as the Radeon RX 580, seem to be working fine on any page size. These cards are available with up to 8GB VRAM. The performance of those cards is more than adequate for many workstation users.

The page size issue should not deter anybody from buying the hardware today. The community is always here to help.

23 February, 2021 10:40PM

February 22, 2021

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

pkgKitten 0.2.1: Now with roxygen2 support


A new release 0.2.1 of pkgKitten hit CRAN earlier today, and has been uploaded to Debian as well. pkgKitten makes it simple to create new R packages via a simple function invocation. A wrapper kitten.r exists in the littler package to make it even easier.

This release builds on the support for tinytest we added in release 0.2.0 by adding more optional support, this time for roxygen2. It also corrects a minor documentation snafu, and updates the CI use.

Changes in version 0.2.1 (2021-02-22)

  • A small documentation error was corrected (David Dalpiaz in #15).

  • A new option ‘bunny’ adds support for roxygen2.

  • Continuous integration now use run.sh from r-ci.

More details about the package are at the pkgKitten webpage, the pkgKitten docs site, and the pkgKitten GitHub repo.

Courtesy of my CRANberries site, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

22 February, 2021 11:26PM

hackergotchi for Daniel Pocock

Daniel Pocock

Master, main and abuse

Free and open source software communities recently spent a lot of time and effort on renaming the master branches in Git repositories to main, or some other name, due to the association of the word master with the horror of slavery.

I plan to tackle the slavery issue in a separate blog. In this blog, my target is the misappropriation of the word abuse.

If we are sincere about abandoning the word master, we also need to stop using the word abuse, except in those situations where it is legitimate to use that word.

Abuse has a clear meaning. In the last week, we've seen women speak up about rape in Australia's parliament and misogyny on the set of Buffy the Vampire Slayer.

Buffy the Vampire Slayer

These are incredibly serious accusations.

The Buffy accusations are remarkably similar to the accusations against Matthias Kirschner, President of FSFE. The free software community elected me as a community representative in that organization. In 2018, after observing the culture of threats and blackmail, I resigned in disgust. Each new revelation about FSFE only confirms that I made the right decision to distance myself from those people.

Yet the accusations from the Australian parliament are even more disturbing. Having visited there on multiple occasions, I couldn't help contemplating the possibility that I may have visited the same office where this crime took place.

In the photo below, there may even be an unintended hint of male entitlement: I'm wandering around Australia's capitol in a t-shirt. It may simply be a reflection of how we live in Australia, the minimum dress code for visiting parliament doesn't set a very high bar. That particular t-shirt isn't easy to come by. The woman on the left is Senator Stott-Despoja, Australia's youngest woman in parliament and subsequently Australia's ambassador for women and girls. How shocking would it be if the crime took place in the same room where we took this photo?

The Debian Project is one of the oldest GNU/Linux distributions. In the 27 years of its existence, so-called leaders have never published a consolidated financial report. When people asked about the Google $300,000 obfuscated by $300,000 from the Handshake Foundation, leaders classified all questions as abuse. When oligarchs behave like this and use the word abuse to deflect questions about accountability, they are trivializing real victims of abuse.

The people who hid that money from the rest of us simply have no right to use the word abuse. Ever.

The situation in Australia's parliament has followed the same path as Debian: rather than resolving the most substantial issue, the employee was terminated on a minor technicality. A most serious act of abuse trivialized by equating it with a bureaucratic misdemeanor.

Natasha Stott-Despoja, Daniel Pocock, Parliament House, Canberra, Australia

Lad culture: when I found a rat in Australia's parliament

Visiting Canberra, I would usually carry my SLR. You never know who (or what) you might meet. After hearing about the bravery of the women speaking up this week and the discussions about the culture problems there, I felt now is the right time to share these images from the billiard hall.

There is nothing political about this blog or the photos. Many Australian men feel ashamed about the way our political leaders appear to live.

Australian Parliament House, Canberra, Billiard Room, Dead Rat

22 February, 2021 09:55PM

Master, main and abuse

Free and open source software communities recently spent a lot of time and effort on renaming the master branches in Git repositories to main, or some other name, due to the association of the word master with the horror of slavery.

I plan to tackle the slavery issue in a separate blog. In this blog, my target is the misappropriation of the word abuse.

If we are sincere about abandoning the word master, we also need to stop using the word abuse, except in those situations where it is legitimate to use that word.

Abuse has a clear meaning. In the last week, we've seen women speak up about rape in Australia's parliament and misogyny on the set of Buffy the Vampire Slayer.

Buffy the Vampire Slayer

These are incredibly serious accusations.

The Buffy accusations are remarkably similar to the accusations against Matthias Kirschner, President of FSFE. The free software community elected me as a community representative in that organization. In 2018, after observing the culture of threats and blackmail, I resigned in disgust. Each new revelation about FSFE only confirms that I made the right decision to distance myself from those people.

Yet the accusations from the Australian parliament are even more disturbing. Having visited there on multiple occasions, I couldn't help contemplating the possibility that I may have visited the same office where this crime took place.

In the photo below, there may even be an unintended hint of male entitlement: I'm wandering around Australia's capitol in a t-shirt. It may simply be a reflection of how we live in Australia, the minimum dress code for visiting parliament doesn't set a very high bar. That particular t-shirt isn't easy to come by. The woman on the left is Senator Stott-Despoja, Australia's youngest woman in parliament and subsequently Australia's ambassador for women and girls. How shocking would it be if the crime took place in the same room where we took this photo?

The Debian Project is one of the oldest GNU/Linux distributions. In the 27 years of its existence, so-called leaders have never published a consolidated financial report. When people asked about the Google $300,000 obfuscated by $300,000 from the Handshake Foundation, leaders classified all questions as abuse. When oligarchs behave like this and use the word abuse to deflect questions about accountability, they are trivializing real victims of abuse.

The people who hid that money from the rest of us simply have no right to use the word abuse. Ever.

The situation in Australia's parliament has followed the same path as Debian: rather than resolving the most substantial issue, the employee was terminated on a minor technicality. A most serious act of abuse trivialized by equating it with a bureaucratic misdemeanor.

Natasha Stott-Despoja, Daniel Pocock, Parliament House, Canberra, Australia

Lad culture: when I found a rat in Australia's parliament

Visiting Canberra, I would usually carry my SLR. You never know who (or what) you might meet. After hearing about the bravery of the women speaking up this week and the discussions about the culture problems there, I felt now is the right time to share these images from the billiard hall.

There is nothing political about this blog or the photos. Many Australian men feel ashamed about the way our political leaders appear to live.

Australian Parliament House, Canberra, Billiard Room, Dead Rat

22 February, 2021 09:55PM

Dima Kogan

feedgnuplot: labelled bar charts and a guide

I just released feedgnuplot 1.57, which includes two new pieces that I've long thought about adding:

Labelled bar charts

I've thought about adding these for a while, but had no specific need for them. Finally, somebody asked for it, and I wrote the code. Now that I can, I will probably use these all the time. The new capability can override the usual numerical tic labels on the x axis, and instead use text from a column in the data stream.

The most obvious use case is labelled bar graphs:

echo "# label value
      aaa     2
      bbb     3
      ccc     5
      ddd     2" | \
feedgnuplot --vnl \
            --xticlabels \
            --with 'boxes fill solid border lt -1' \
            --ymin 0 --unset grid


But the usage is completely generic. All --xticlabels does, is to accept a data column as labels for the x-axis tics. Everything else that's supported by feedgnuplot and gnuplot works as before. For instance, I can give a domain, and use a style that takes y values and a color:

echo "# x label y color
        5 aaa   2 1
        6 bbb   3 2
       10 ccc   5 4
       11 ddd   2 1" | \
feedgnuplot --vnl --domain \
            --xticlabels \
            --tuplesizeall 3 \
            --with 'points pt 7 ps 2 palette' \
            --xmin 4 --xmax 12 \
            --ymin 0 --ymax 6 \
            --unset grid


And we can use gnuplot's support for clustered histograms:

echo "# x label a b
        5 aaa   2 1
        6 bbb   3 2
       10 ccc   5 4
       11 ddd   2 1" | \
vnl-filter -p label,a,b | \
feedgnuplot --vnl \
            --xticlabels \
            --set 'style data histogram' \
            --set 'style histogram cluster gap 2' \
            --set 'style fill solid border lt -1' \
            --autolegend \
            --ymin 0 --unset grid


Or we can stack the bars on top of one another:

echo "# x label a b
        5 aaa   2 1
        6 bbb   3 2
       10 ccc   5 4
       11 ddd   2 1" | \
vnl-filter -p label,a,b | \
feedgnuplot --vnl \
            --xticlabels \
            --set 'style data histogram' \
            --set 'style histogram rowstacked' \
            --set 'boxwidth 0.8' \
            --set 'style fill solid border lt -1' \
            --autolegend \
            --ymin 0 --unset grid


This is gnuplot's "row stacking". It also supports "column stacking", which effectively transposes the data, and it's not obvious to me that makes sense in the context of feedgnuplot. Similarly, it can label y and/or z axes; I can't think of a specific use case, so I don't have a realistic usage in mind, and I don't support that yet. If anybody can think of a use case, email me.

Notes and limitations:

  • Since with --domain you can pass in both an x value and a tic label, it is possible to give it conflicting tic labels for the same x value. gnuplot itself has this problem too, and it just takes the last label it has for a given x. This is probably good-enough.
  • feedgnuplot uses whitespace-separated columns with no escape mechanism, so the field labels cannot have whitespace in it. Fixing this is probably not worth the effort.
  • These tic labels do not count towards the tuplesize
  • I really need to add a similar feature to gnuplotlib. This will happen when I need it or when somebody asks for it, whichever comes first.

A feedgnuplot guide

This fills in a sorely needed missing part of the documentation: the main feedgnuplot website now has a page containing examples and corresponding graphical output. This serves as a tutorial and a gallery demonstrating some usages. It's somewhat incomplete, since it can't show streaming plots, or real-world interfacing with stuff that produces data: some of those usages remain the the README. It's a million times better than what I had before though, which was nothing.

Internally this is done just like the gnuplotlib guide: the thing is an org-mode document with org-babel snippets that are evaluated by emacs to make the images. There's some fancy emacs lisp to tie it all together. Works great!

22 February, 2021 06:12PM by Dima Kogan

hackergotchi for Charles Plessy

Charles Plessy


I was using a container for a bioinformatics tool released two weeks ago, but my shell script wrapping the tools could not run because the container was built around an old version of Debian (Jessie) that was released in 2015. I was asked to use a container for bioinformatics, based on conda, and found one that distributes coreutils, but it did not include a real version of sed. I try Debian's docker image. No luck; it does not contain ps, which my workflow manager needs. But fortunately I eventually figured out that Ubuntu's Docker image contains coreutils, sed and ps together! In the world of containers, this sounds like a little miracle.

22 February, 2021 04:11PM

John Goerzen

Recovering Our Lost Free Will Online: Tools and Techniques That Are Available Now

As I’ve been thinking and writing about privacy and decentralization lately, I had a conversation with a colleague this week, and he commented about how loss of privacy is related to loss of agency: that is, loss of our ability to make our own choices, pursue our own interests, and be master of our own attention.

In terms of telecommunications, we have never really been free, though in terms of Internet and its predecessors, there have been times where we had a lot more choice. Many are too young to remember this, and for others, that era is a distant memory.

The irony is that our present moment is one of enormous consolidation of power, and yet also one of a proliferation of technologies that let us wrest back some of that power. In this post, I hope to enlighten or remind us of some of the choices we have lost — and also talk about the ways in which we can choose to regain them, already, right now.

I will talk about the possibilities, the big dreams that are possible now, and then go into more detail about the solutions.

The Problems & Possibilities

The limitations of “online”

We make the assumption that we must be “online” to exchange data. This is reinforced by many “modern” protocols; Twitter clients, for instance, don’t tend to let you make posts by relaying them through disconnected devices.

What would it be like if you could fully participate in global communities without a constant Internet connection? If you could share photos with your friends, read the news, read your email, etc. even if you don’t have a connection at present? Even if the device you use to do that never has a connection, but can route messages via other devices that do?

Would it surprise you to learn that this was once the case? Back in the days of UUCP, much email and Usenet news — a global discussion forum that didn’t require an Internet connection — was relayed via occasional calls over phone lines. This technology remains with us, and has even improved.

Sadly, many modern protocols make no effort in this regard. Some email clients will let you compose messages offline to send when you get online later, but the assumption always is that you will be connected to an IP network again soon.

NNCP, on the other hand, lets you relay messages over TCP, a radio, a satellite, or a USB stick. Email and Usenet, since they were designed in an era where store-and-forward was valued, can actually still be used in an entirely “offline” fashion (without ever touching an IP-based network). All it takes is for someone to care to make it happen. You can even still do it over UUCP if you like.

The physical and data link layers

Many of us just accept that we communicate in a few ways: Wifi for short distances, and then cable modems or DSL for our local Internet connection, and then many people are fuzzy about what happens after that. Or, alternatively, we have 4G phones that are the local Internet connection, and the same “fuzzy” things happen after.

Think about this for a moment. Which of these do you control in any way? Sometimes just wifi, sometimes maybe you have choices of local Internet providers. After that, your traffic is handled by enormous infrastructure companies.

There is choice here.

People in ham radio have been communicating digitally over long distances without the support of the traditional Internet for decades, but the technology to do this is now more accessible to anyone. Long-distance radio has had tremendous innovation in the last decade; cheap radios can now communicate over several miles/km without any other infrastructure at all. We all carry around radios (Wifi and Bluetooth) in our pockets that don’t have to be used as mere access points to the Internet or as drivers of headphones, but can also form their own networks directly (Briar).

Meshtastic is an example; it’s an instant messenger that can form a mesh over many miles/km and requires no IP infrastructure at all. Briar is similar. XBee radios form a mesh in hardware, allowing peers to reach each other (also over many miles/km) with a serial or framed protocol.

Loss of peer-to-peer

Back in the late 90s, I worked at a university. I had a 386 on my desk for a workstation – not a powerful computer even then. But I put the boa webserver on it and could just serve pages on the Internet. I didn’t have to get permission. Didn’t have to pay a hosting provider. I could just DO it.

And of course that is because the university had no firewall and no NAT. Every PC at the university was a full participant on the Internet as much as the servers at Microsoft or DEC. All I needed was a DNS entry. I could run my own SMTP server if I wanted, run a web or Gopher server, and that was that.

There are many reasons why this changed. Nowadays most residential ISPs will block SMTP for their customers, and if they didn’t, others would; large email providers have decided not to federate with IPs in residential address spaces. Most people have difficulty even getting a static IP address in the first place. Many are behind firewalls, NATs, or both, meaning that incoming connections of any kind are problematic.

Do you see what that means? It has weakened the whole point of the Internet being a network of peers. While IP still acts that way, as a practical matter, there are clients that are prevented from being servers by administrative policy they have no control over.

Imagine if you, a person with an Internet connection to your laptop or phone, could just decide to host a website, or a forum on it. For moderate levels of load, they are certainly capable of this. The only thing in the way is the network management policies you can’t control.

Elaborate technologies exist to try to bridge this divide, and some, like Tor or cjdns, can work quite well. More on this below.

Expense of running something popular

Related to the loss of peer-to-peer infrastructure is the very high cost of hosting something popular. Do you want to share videos with lots of people? That almost certainly is going to require expensive equipment and bandwidth.

There is a reason that there are only a small handful of popular video streaming sites online. It requires a ton of money to host videos at scale.

What if it didn’t? What if you could achieve economies of scale so much that you, an individual, could compete with the likes of YouTube? You wouldn’t necessarily have to run ads to support the service. You wouldn’t have to have billions of dollars or billions of viewers just to make it work.

This technology exists right now. Of course many of you are aware of how Bittorrent leverages the swarm for files. But projects like IPFS, Dat, and Peertube have taken this many steps further to integrate it into a global ecosystem. And, at least in the case of Peertube, this is a thing that works right now in any browser already!

Application-level “walled gardens”

I was recently startled at how much excitement there was when Github introduced “dark mode”. Yes, Github now offers two colors on its interface. Already back in the 80s and 90s, many DOS programs had more options than that.

Git is a decentralized protocol, but Github has managed to make it centralized.

Email is a decentralized protocol — pick your own provider, and they all communicate — but Facebook and Twitter aren’t. You can’t just pick your provider for Facebook. It’s Facebook or nothing.

There is a profit motive in locking others out; these networks want to keep you using their platforms because their real customers are advertisers, and they want to keep showing you ads.

Is it possible to have a world where you get to pick your own app for sharing photos, and it works even if your parents use a different one? Yes, yes it is.

Mastodon and the Fediverse are fantastic examples for social media. Pixelfed is specifically designed for photos, Mastodon for short-form communication, there’s Pleroma for more long-form communication, and they all work together. You can use Mastodon to read Pleroma content or look at Pixelfed photos, and there are many (free) providers of each.

Freedom from manipulation

I recently wrote about the dangers of the attention economy, so I won’t go into a lot of detail here. Fundamentally, you are not the customer of Facebook or Google; advertisers are. They optimize their site to keep you on it as much as possible so that they can show you as many ads as possible which makes them as much money as possible. Ads, of course, are fundamentally seeking to manipulate your behavior (“buy this product”).

By lowering the cost of running services, we can give a huge boost to hobbyists and nonprofits that want to do so without an ultimate profit motive. For-profit companies benefit also, with a dramatically reduced cost structure that frees them to pursue their mission instead of so many ads.

Freedom from snooping (privacy and anonymity)

These days, it’s not just government snooping that people think about. It’s data stolen by malware, spies at corporations (whether human or algorithmic), and even things like basic privacy of one’s own security footage. Here the picture is improving; encryption in transit, at least at a basic level, has become much more common with TLS being a standard these days. Sadly, end-to-end encryption (E2EE) is not nearly as much, perhaps because corporations have a profit motive to have access to your plaintext and metadata.

Closely related to privacy is anonymity: that is, being able to do things in an anonymous fashion. The two are not necessarily equal: you could send an encrypted message but reveal who the correspondents are, as with email; or, you could send a plaintext message over a Tor exit node that hides who the correspondents are. It is sometimes difficult to achieve both.

Nevertheless, numerous answers exist here that tackle one or both problems, from the Signal messenger to Tor.

Solutions That Exist Today

Let’s dive in to some of the things that exist today.

One concept you’ll see in many of these is integrated encryption with public keys used for addressing. In other words, your public key is akin to an IP address (and in some cases, is literally your IP address.)

Data link and networking technologies (some including P2P)

  • Starting with the low-power and long-distance technologies, I’ve written quite a bit about LoRA, which are low-power long-distance radios. They can easily achieve several miles/km while still using much less than 1W of power. LoRA is a common building block of mesh off-the-grid messenger systems such as meshtastic, which forms an ad-hoc mesh of LoRA devices with days-long battery life and miles-long communication abilities. LoRA trades speed for bandwidth; in its longest-distance modes, it may operate at 300bps or less. That is not a typo. Some LoRAWAN devices have battery life measured in years (usually one-way sensors and such). Also, the Pine64 folks are working to integrate LoRA on nearly all their product line, which includes single-board computers, phones, and laptops.
  • Similar to LoRA is XBee SX from Digi. While not quite as long-distance as LoRA, it does still do quite a bit with low power and also goes many miles. XBee modules have automatic mesh routing in firmware, and can be used in either frame mode or “serial cable emulation” mode in which they act as if they’re a serial cable. Unlike plain LoRA, XBee radios do hardware retransmit. They also run faster, at up to about 150Kbps – though that is still a lot slower than wifi.
  • I’ve written about secure mesh messengers recently. One of them, Briar, particularly stands out in that it is able to form an ad-hoc mesh using phone’s Bluetooth radios. It can also route messages over the public Internet, which it does exclusively using Tor.
  • I’ve also written a lot about NNCP, the sort of modernized UUCP. NNCP is completely different than the others here in that it is a store-and-forward network – sort of a modern UUCP. NNCP has easy built-in support for routing packets using USB drives, clean serial interfaces, TCP, basically anything you can pipe to, even broadcast satellite and such. And you don’t even have to pick one; you can use all of the above: Internet when it’s available, USB sticks or portable hard drives when not, etc. It uses Tor-line onion routing with E2EE. You’re not going to run TCP over NNCP, but files (including videos), backups, email, even remote execution are all possible. It is the most “Unixy” of the modern delay-tolerant networks and makes an excellent choice for a number of use cases where store-and-forward and extreme flexibility in transportation make a lot of sense.
  • Moving now into the range of speeds and technologies we’re more used to, there is a lot of material out there on building mesh networks on Wifi or Wifi-adjacent technology. Amateur radio operators have been active in this area for years, and even if you aren’t a licensed ham and don’t necessarily flash amateur radio firmware onto your access points, a lot of the ideas and concepts they cover could be of interest. For instance, the Amateur Radio Emergency Data Network covers both permanent and ad-hoc meshs, and this AREDN video covers device selection for AREDN — which also happens to be devices that would be useful for quite a few other mesh or long-distance point-to-point setups.
  • Once you have a physical link of some sort, cjdns and the Hyperboria network have the goals of literally replacing the Internet – but are fully functional immediately. cjdns assigns each node an IPv6 address based on its public key. The network uses DHT for routing between nodes. It can run directly atop Ethernet (and Wifi) as its own native protocol, without an IP stack underneath. It can also run as a layer atop the current Internet. And it can optionally be configured to let nodes find an exit node to reach the current public Internet, which they can do opportunistically if given permission. All traffic is E2EE. One can run an isolated network, or join the global Hyperboria network. The idea is that local meshes could be formed, and then geographically distant meshes can be linked together by simply using the current public Internet as a dumb transport. This, actually, strongly resembles the early days of Internet buildout under NSFNet. The Torento Mesh is a prominent user of cjdns, and they publish quite a bit of information online. cjdns as a standalone identity is in decline, but forms the basis of the pkt network, which is designed to foster an explosion in WISPs.
  • Similar in concept to cjdns is Yggdrasil, which uses a different routing algorithm. It is now more active than cjdns and has active participants and developers.
  • Althea is a startup in this space, hoping to encourage communities to build meshes whose purpose is to provide various routes to access to the traditional Internet, including digital currency micropayments. This story documents how one rural community is using it.
  • Tor is a somewhat interesting case. While it doesn’t provide kernel-level routing, it does provide a SOCKS5 proxy. Traditionally, Tor is used to achieve anonymity while browsing the public Internet via an exit node. However, you can stay entirely in-network by using onion services (basically ports that are open to Tor). All Tor traffic is onion-routed so that the originating IP cannot be discovered. Data within Tor is E2EE, though if you are using an exit node to the public Internet, that of course can’t apply there.
  • GNUnet is a large suite of tools for P2P communication. It includes file downloading, Tor-like IP over the network, a DNS replacement, and facilitates quite a few of the goals discussed here. (Added in a 2021-02-22 update)

P2P Infrastructure

While some of the technologies above, such as cjdns, explicitly facitilitate peer-to-peer communication, there are some other application-level technologies to look at.

  • IPFS has been having a lot of buzz lately, since the Brave browser integrated support. IPFS headlines as “powers the distributed web”, but it is actually more than that; various other apps layer atop it. The core idea is that content you request gets reshared by your node for some period of time, somewhat akin to Bittorrent. IPFS runs atop the regular Internet and is typically accessed through an app.
  • The Dat Protocol is somewhat similar in concept to IPFS, though the approach is somewhat different; it emphasizes efficient distribution of updates at the expense of requiring a git-like history.
  • IPFS itself is based on libp2p, which is designed to be a generic infrastructure for adding P2P capabilities to your own code. It is probably fair to say libp2p is still quite complex compared to ordinary TCP, and the language support is in its infancy, but nevertheless it is quite an exciting development to watch.
  • Of course almost all of us are familiar with Bittorrent, the software that first popularized the idea of a distributed mesh sharing knowledge about which chunks of a dataset they have in order to maximize the efficiency of distributing the whole thing. Bittorrent is still in wide use (and, despite its reputation, that wide use includes legitimate users such as archive.org and Debian).
  • I recently wrote about building a delay-tolerant offline-capable mesh with Syncthing. Syncthing, on its surface, is something like an open source Dropbox. But look into a bit and you realize it’s fully P2P, serverless, can support various network topologies including intermittent connectivity between network parts, and such. My article dives into that in more detail. If your needs are mostly related to files, Syncthing can make a fine mesh infrastructure that is auto-healing and is equally at home on the public Internet, a local wifi access point with no Internet at all, a private mesh like cjdns, etc.
  • Also showing some promise is Secure Scuttlebutt (SSB). Its most well-known application is a social network, but in my opinion some of the other applications atop SSB are more interesting. SSB is designed to be offline-friendly, can do things like automatically exchange data with peers on the same Wifi (eg, a coffee shop), etc., though it is an append-only log that can be unwieldy on mobile sometimes.

Instant Messengers and Chat

I won’t go into a lot of detail here since I recently wrote a roundup of secure mesh messengers and also a followup article about Signal and some hidden drawbacks of P2P. Please refer to those articles for some interesting things that are happening in this space.

Matrix is a distributed IM platform similar in concept to Slack or IRC, but globally distributed in a mesh. It supports optional E2EE.

Social Media

I wrote recently about how to join the Fediverse, which covered joining Mastodon, a federeated, decentralized social network. Mastodon is the largest of these, with several million users, and is something of a much nicer version of Twitter.

Mastodon is also part of what is known as the “Fediverse”, which are applications that are loosely joined together by their support of the ActivityPub protocol. Other popular Fediverse applications include Pixelfed (similar to Instagram) and Peertube for sharing video. Peertube is particularly interesting in that it supports Webtorrent for efficiently distributing popular videos. Webtorrent is akin to Bittorrent running efficiently inside your browser.

Concluding Remarks

Part of my goal with this is encouraging people to dream big, to ask questions like:

What could you do if offline were easy?

What is possible if you have freedom in the physical and data link layers? Dream big.

We’re so used to thinking that it’s quite difficult for two devices on the Internet to talk to each other. What would be possible if this were actually quite easy?

The assumption that costs rise dramatically as popularity increases is also baked into our thought processes. What if that weren’t the case — could you take on Youtube from your garage? Would lowering barriers to entry lower the ad economy and let nonprofits have more equal footing with large corporations?

We have so many walled gardens, from Github to Facebook, that we almost forget it doesn’t have to be that way.

So having asked these questions, my secondary point is to suggest that these aren’t pie-in-the-sky notions. These possibilites are with us right now.

You’ll notice from this list that virtually every one of these technologies is ad-free at its heart (though some would be capable of serving ads). They give you back your attention. Many preserve privacy, anonymity, or both. Many dramatically improve your freedom of association and communication. Technologies like IPFS and Bittorrent ease the burden of running something popular.

Some are quite easy to use (Mastodon or Peertube) while others are much more complex (libp2p or the lower-level mesh network systems).

Clearly there is still room for improvement in many areas.

But my fundamental point is this: good technology is here, right now. Technical people can vote with their feet and wallets and start using it. Early adopters will help guide the way for the next set of improvements. Join us!

22 February, 2021 04:13AM by John Goerzen

Russ Allbery

Review: Finder

Review: Finder, by Suzanne Palmer

Series: Finder Chronicles #1
Publisher: DAW Books
Copyright: 2019
ISBN: 0-7564-1511-X
Format: Kindle
Pages: 391

Fergus Ferguson is a repo man, or professional finder as he'd prefer. He locates things taken by people who don't own them and returns them to their owners. In this case, the thing in question is a sentient starship, and the person who stole it is Arum Gilger, a warlord in a wired-together agglomeration of space habitats and mined-out asteroids named Cernekan. Cernee, as the locals call it, is in the backwaters of human space near the Gap between the spiral arms of the galaxy.

One of Fergus's first encounters in Cernee is with an old lichen farmer named Mattie Vahn who happens to take the same cable car between stations that he does. Bad luck for Fergus, since that's also why Gilger's men first disable and then blow up the cable car, leaving Mattie dead and Fergus using the auto-return feature of Mattie's crates to escape to the Vahns' home station. The Vahns are not a power in Cernee, not exactly, but they do have some important alliances and provide an inroads for Fergus to get the lay of the land and map out a plan to recover the Venetia's Sword.

This is a great hook. I would happily read a whole series about an interstellar repo man, particularly one like Fergus who only works for the good guys and recovers things from petty warlords. Fergus is a thoughtful, creative loner whose style is improvised plans, unexpected tactics, and thinking on his feet rather than either bluster or force (although there is a fair bit of death in this book, some of which is gruesome). About two-thirds of the book is in roughly that expected shape. Fergus makes some local contacts, maps out the political terrain, and maneuvers himself towards his target through a well-crafted slum of wired-together habitats and working-class miners. Also, full points for the creative security system on the starship that tries to solve a nearly impossible problem (a backdoor supplementing pre-shared keys with a cultural authentication scheme that can't be vulnerable to brute force or database searches).

Halfway through, though, Palmer throws a curve ball at the reader that involves the unexplained alien presence that's been lurking around the system. That part of the plot shifts focus somewhat abruptly from the local power struggle Fergus has been navigating to something far more intrusive and personal. Fergus has to both reckon with a drastic change in his life and deal with memories of his early life on an Earth drowning in climate change, his abusive childhood, and his time spent in the Martian resistance.

This is also a fine topic for an SF novel, but I think Finder suffered a bit from falling between two stools. The fun competence drama of the lone repossession agent striking back against petty tyrants by taking away their toys is derailed by the sudden burst of introspection and emotional processing, but the processing is not deep or complex enough to carry the story on its own. Fergus had an awful and emotionally alienated childhood followed by some nasty trauma, to which he has responded by carefully never getting close to anyone so that he never hurts anyone who relies on him. And yet, he's a fundamentally decent person and makes friends despite himself, and from there you can probably write the rest of the arc yourself. There's nothing wrong with this as the emotional substrate of a book that's primarily focused on an action plot, but the screeching change of focus threw me off.

The good news is that the end of the book returns to the bits that I liked about the first half. The mixed news is that I thought the political situation in Cernee resolved much too easily and much too straightforwardly. I would have preferred the twisty alliances to stay twisty, rather than collapse messily into a far simpler moral duality. I will also speak on behalf of all the sentient starship lovers out there and complain that the Venetia's Sword was woefully underused. It had better show up in a future volume!

This unsteadiness and a few missed opportunities make Finder a good book rather than a great one, but I was still happily entertained and willing to write that off as first-novel unevenness. There are a lot of background elements left unresolved for a future volume, but Finder comes to a satisfying conclusion. Recommended if you're looking for an undemanding space action story with a quick pace and decent, if not very deep, characters.

Followed by Driving the Deep.

Rating: 7 out of 10

22 February, 2021 04:06AM

February 21, 2021

Enrico Zini

Dmitry Shachnev

ReText turns 10 years

Exactly ten years ago, in February 2011, the first commit in ReText git repository was made. It was just a single 364 lines Python file back then (now the project has more than 6000 lines of Python code).

Since 2011, the editor migrated from SourceForge to GitHub, gained a lot of new features, and — most importantly — now there is an active community around it, which includes both long-time contributors and newcomers who create their first issues or pull requests. I don’t always have enough time to reply to issues or implement new features myself, but the community members help me with this.

Earlier this month, I made a new release (7.2), which adds a side panel with directory tree (contributed by Xavier Gouchet), option to fully highlight wrapped lines (contributed by nihillum), ability to search in the preview mode and much more — see the release page on GitHub.

Side panel in ReText

Also a new version of PyMarkups module was released, which contains all the code for processing various markup languages. It now supports markdown-extensions.yaml files which allow specifying complex extensions options and adds initial support for MathJax 3.

Also check out the release notes for 7.1 which was not announced on this blog.

Future plans include making at least one more release this year, adding support for Qt 6. Qt 5 support will last for at least one more year.

21 February, 2021 06:30PM by Dmitry Shachnev

hackergotchi for David Bremner

David Bremner

git-annex and ikiwiki, not as hard as I expected


So apparently there's this pandemic thing, which means I'm teaching "Alternate Delivery" courses now. These are just like online courses, except possibly more synchronous, definitely less polished, and the tuition money doesn't go to the College of Extended Learning. I figure I'll need to manage share videos, and our learning management system, in the immortal words of Marie Kondo, does not bring me joy. This has caused me to revisit the problem of sharing large files in an ikiwiki based site (like the one you are reading).

My goto solution for large file management is git-annex. The last time I looked at this (a decade ago or so?), I was blocked by git-annex using symlinks and ikiwiki ignoring them for security related reasons. Since then two things changed which made things relatively easy.

  1. I started using the rsync_command ikiwiki option to deploy my site.

  2. git-annex went through several design iterations for allowing non-symlink access to large files.


In my ikiwiki config

    # attempt to hardlink source files? (optimisation for large files)
    hardlink => 1,

In my ikiwiki git repo

$ git annex init
$ git annex add foo.jpg
$ git commit -m'add big photo'
$ git annex adjust --unlock                 # look ikiwiki, no symlinks
$ ikiwiki --setup ~/.config/ikiwiki/client  # rebuild my local copy, for review
$ ikiwiki --setup /home/bremner/.config/ikiwiki/rsync.setup --refresh  # deploy

You can see the result at photo

21 February, 2021 01:47PM

hackergotchi for Matthew Garrett

Matthew Garrett

Making hibernation work under Linux Lockdown

Linux draws a distinction between code running in kernel (kernel space) and applications running in userland (user space). This is enforced at the hardware level - in x86-speak[1], kernel space code runs in ring 0 and user space code runs in ring 3[2]. If you're running in ring 3 and you attempt to touch memory that's only accessible in ring 0, the hardware will raise a fault. No matter how privileged your ring 3 code, you don't get to touch ring 0.

Kind of. In theory. Traditionally this wasn't well enforced. At the most basic level, since root can load kernel modules, you could just build a kernel module that performed any kernel modifications you wanted and then have root load it. Technically user space code wasn't modifying kernel space code, but the difference was pretty semantic rather than useful. But it got worse - root could also map memory ranges belonging to PCI devices[3], and if the device could perform DMA you could just ask the device to overwrite bits of the kernel[4]. Or root could modify special CPU registers ("Model Specific Registers", or MSRs) that alter CPU behaviour via the /dev/msr interface, and compromise the kernel boundary that way.

It turns out that there were a number of ways root was effectively equivalent to ring 0, and the boundary was more about reliability (ie, a process running as root that ends up misbehaving should still only be able to crash itself rather than taking down the kernel with it) than security. After all, if you were root you could just replace the on-disk kernel with a backdoored one and reboot. Going deeper, you could replace the bootloader with one that automatically injected backdoors into a legitimate kernel image. We didn't have any way to prevent this sort of thing, so attempting to harden the root/kernel boundary wasn't especially interesting.

In 2012 Microsoft started requiring vendors ship systems with UEFI Secure Boot, a firmware feature that allowed[5] systems to refuse to boot anything without an appropriate signature. This not only enabled the creation of a system that drew a strong boundary between root and kernel, it arguably required one - what's the point of restricting what the firmware will stick in ring 0 if root can just throw more code in there afterwards? What ended up as the Lockdown Linux Security Module provides the tooling for this, blocking userspace interfaces that can be used to modify the kernel and enforcing that any modules have a trusted signature.

But that comes at something of a cost. Most of the features that Lockdown blocks are fairly niche, so the direct impact of having it enabled is small. Except that it also blocks hibernation[6], and it turns out some people were using that. The obvious question is "what does hibernation have to do with keeping root out of kernel space", and the answer is a little convoluted and is tied into how Linux implements hibernation. Basically, Linux saves system state into the swap partition and modifies the header to indicate that there's a hibernation image there instead of swap. On the next boot, the kernel sees the header indicating that it's a hibernation image, copies the contents of the swap partition back into RAM, and then jumps back into the old kernel code. What ensures that the hibernation image was actually written out by the kernel? Absolutely nothing, which means a motivated attacker with root access could turn off swap, write a hibernation image to the swap partition themselves, and then reboot. The kernel would happily resume into the attacker's image, giving the attacker control over what gets copied back into kernel space.

This is annoying, because normally when we think about attacks on swap we mitigate it by requiring an encrypted swap partition. But in this case, our attacker is root, and so already has access to the plaintext version of the swap partition. Disk encryption doesn't save us here. We need some way to verify that the hibernation image was written out by the kernel, not by root. And thankfully we have some tools for that.

Trusted Platform Modules (TPMs) are cryptographic coprocessors[7] capable of doing things like generating encryption keys and then encrypting things with them. You can ask a TPM to encrypt something with a key that's tied to that specific TPM - the OS has no access to the decryption key, and nor does any other TPM. So we can have the kernel generate an encryption key, encrypt part of the hibernation image with it, and then have the TPM encrypt it. We store the encrypted copy of the key in the hibernation image as well. On resume, the kernel reads the encrypted copy of the key, passes it to the TPM, gets the decrypted copy back and is able to verify the hibernation image.

That's great! Except root can do exactly the same thing. This tells us the hibernation image was generated on this machine, but doesn't tell us that it was done by the kernel. We need some way to be able to differentiate between keys that were generated in kernel and ones that were generated in userland. TPMs have the concept of "localities" (effectively privilege levels) that would be perfect for this. Userland is only able to access locality 0, so the kernel could simply use locality 1 to encrypt the key. Unfortunately, despite trying pretty hard, I've been unable to get localities to work. The motherboard chipset on my test machines simply doesn't forward any accesses to the TPM unless they're for locality 0. I needed another approach.

TPMs have a set of Platform Configuration Registers (PCRs), intended for keeping a record of system state. The OS isn't able to modify the PCRs directly. Instead, the OS provides a cryptographic hash of some material to the TPM. The TPM takes the existing PCR value, appends the new hash to that, and then stores the hash of the combination in the PCR - a process called "extension". This means that the new value of the TPM depends not only on the value of the new data, it depends on the previous value of the PCR - and, in turn, that previous value depended on its previous value, and so on. The only way to get to a specific PCR value is to either (a) break the hash algorithm, or (b) perform exactly the same sequence of writes. On system reset the PCRs go back to a known value, and the entire process starts again.

Some PCRs are different. PCR 23, for example, can be reset back to its original value without resetting the system. We can make use of that. The first thing we need to do is to prevent userland from being able to reset or extend PCR 23 itself. All TPM accesses go through the kernel, so this is a simple matter of parsing the write before it's sent to the TPM and returning an error if it's a sensitive command that would touch PCR 23. We now know that any change in PCR 23's state will be restricted to the kernel.

When we encrypt material with the TPM, we can ask it to record the PCR state. This is given back to us as metadata accompanying the encrypted secret. Along with the metadata is an additional signature created by the TPM, which can be used to prove that the metadata is both legitimate and associated with this specific encrypted data. In our case, that means we know what the value of PCR 23 was when we encrypted the key. That means that if we simply extend PCR 23 with a known value in-kernel before encrypting our key, we can look at the value of PCR 23 in the metadata. If it matches, the key was encrypted by the kernel - userland can create its own key, but it has no way to extend PCR 23 to the appropriate value first. We now know that the key was generated by the kernel.

But what if the attacker is able to gain access to the encrypted key? Let's say a kernel bug is hit that prevents hibernation from resuming, and you boot back up without wiping the hibernation image. Root can then read the key from the partition, ask the TPM to decrypt it, and then use that to create a new hibernation image. We probably want to prevent that as well. Fortunately, when you ask the TPM to encrypt something, you can ask that the TPM only decrypt it if the PCRs have specific values. "Sealing" material to the TPM in this way allows you to block decryption if the system isn't in the desired state. So, we define a policy that says that PCR 23 must have the same value at resume as it did on hibernation. On resume, the kernel resets PCR 23, extends it to the same value it did during hibernation, and then attempts to decrypt the key. Afterwards, it resets PCR 23 back to the initial value. Even if an attacker gains access to the encrypted copy of the key, the TPM will refuse to decrypt it.

And that's what this patchset implements. There's one fairly significant flaw at the moment, which is simply that an attacker can just reboot into an older kernel that doesn't implement the PCR 23 blocking and set up state by hand. Fortunately, this can be avoided using another aspect of the boot process. When you boot something via UEFI Secure Boot, the signing key used to verify the booted code is measured into PCR 7 by the system firmware. In the Linux world, the Shim bootloader then measures any additional keys that are used. By either using a new key to tag kernels that have support for the PCR 23 restrictions, or by embedding some additional metadata in the kernel that indicates the presence of this feature and measuring that, we can have a PCR 7 value that verifies that the PCR 23 restrictions are present. We then seal the key to PCR 7 as well as PCR 23, and if an attacker boots into a kernel that doesn't have this feature the PCR 7 value will be different and the TPM will refuse to decrypt the secret.

While there's a whole bunch of complexity here, the process should be entirely transparent to the user. The current implementation requires a TPM 2, and I'm not certain whether TPM 1.2 provides all the features necessary to do this properly - if so, extending it shouldn't be hard, but also all systems shipped in the past few years should have a TPM 2, so that's going to depend on whether there's sufficient interest to justify the work. And we're also at the early days of review, so there's always the risk that I've missed something obvious and there are terrible holes in this. And, well, given that it took almost 8 years to get the Lockdown patchset into mainline, let's not assume that I'm good at landing security code.

[1] Other architectures use different terminology here, such as "supervisor" and "user" mode, but it's broadly equivalent
[2] In theory rings 1 and 2 would allow you to run drivers with privileges somewhere between full kernel access and userland applications, but in reality we just don't talk about them in polite company
[3] This is how graphics worked in Linux before kernel modesetting turned up. XFree86 would just map your GPU's registers into userland and poke them directly. This was not a huge win for stability
[4] IOMMUs can help you here, by restricting the memory PCI devices can DMA to or from. The kernel then gets to allocate ranges for device buffers and configure the IOMMU such that the device can't DMA to anything else. Except that region of memory may still contain sensitive material such as function pointers, and attacks like this can still cause you problems as a result.
[5] This describes why I'm using "allowed" rather than "required" here
[6] Saving the system state to disk and powering down the platform entirely - significantly slower than suspending the system while keeping state in RAM, but also resilient against the system losing power.
[7] With some handwaving around "coprocessor". TPMs can't be part of the OS or the system firmware, but they don't technically need to be an independent component. Intel have a TPM implementation that runs on the Management Engine, a separate processor built into the motherboard chipset. AMD have one that runs on the Platform Security Processor, a small ARM core built into their CPU. Various ARM implementations run a TPM in Trustzone, a special CPU mode that (in theory) is able to access resources that are entirely blocked off from anything running in the OS, kernel or otherwise.

comment count unavailable comments

21 February, 2021 08:37AM

Russ Allbery

Review: The Fated Sky

Review: The Fated Sky, by Mary Robinette Kowal

Series: Lady Astronaut #2
Publisher: Tor
Copyright: August 2018
ISBN: 0-7653-9893-1
Format: Kindle
Pages: 380

The Fated Sky is a sequel to The Calculating Stars, but you could start with this book if you wanted to. It would be obvious you'd missed a previous book in the series, and some of the relationships would begin in medias res, but the story is sufficiently self-contained that one could puzzle through.

Mild spoilers follow for The Calculating Stars, although only to the extent of confirming that book didn't take an unexpected turn, and nothing that wouldn't already be spoiled if you had read the short story "The Lady Astronaut of Mars" that kicked this series off. (The short story takes place well after all of the books.) Also some minor spoilers for the first section of the book, since I have to talk about its outcome in broad strokes in order to describe the primary shape of the novel.

In the aftermath of worsening weather conditions caused by the Meteor, humans have established a permanent base on the Moon and are preparing a mission to Mars. Elma is not involved in the latter at the start of the book; she's working as a shuttle pilot on the Moon, rotating periodically back to Earth. But the political situation on Earth is becoming more tense as the refugee crisis escalates and the weather worsens, and the Mars mission is in danger of having its funding pulled in favor of other priorities. Elma's success in public outreach for the space program as the Lady Astronaut, enhanced by her navigation of a hostage situation when an Earth re-entry goes off course and is met by armed terrorists, may be the political edge supporters of the mission need.

The first part of this book is the hostage situation and other ground-side politics, but the meat of this story is the tense drama of experimental, pre-computer space flight. For those who aren't familiar with the previous book, this series is an alternate history in which a huge meteorite hit the Atlantic seaboard in 1952, potentially setting off runaway global warming and accelerating the space program by more than a decade. The Calculating Stars was primarily about the politics surrounding the space program. In The Fated Sky, we see far more of the technical details: the triumphs, the planning, and the accidents and other emergencies that each could be fatal in an experimental spaceship headed towards Mars. If what you were missing from the first book was more technological challenge and realistic detail, The Fated Sky delivers. It's edge-of-your-seat suspenseful and almost impossible to put down.

I have more complicated feelings about the secondary plot. In The Calculating Stars, the heart of the book was an incredibly well-told story of Elma learning to deal with her social anxiety. That's still a theme here but a lesser one; Elma has better coping mechanisms now. What The Fated Sky tackles instead is pervasive sexism and racism, and how Elma navigates that (not always well) as a white Jewish woman.

The centrality of sexism is about the same in both books. Elma's public outreach is tied closely to her gender and starts as a sort of publicity stunt. The space program remains incredibly sexist in The Fated Stars, something that Elma has to cope with but can't truly fix. If you found the sexism in the first book irritating, you're likely to feel the same about this installment.

Racism is more central this time, though. In The Calculating Stars, Elma was able to help make things somewhat better for Black colleagues. She has a much different experience in The Fated Stars: she ends up in a privileged position that hurts her non-white colleagues, including one of her best friends. The merits of taking a stand on principle are ambiguous, and she chooses not to. When she later tries to help Black astronauts, she does so in a way that's focused on her perceptions rather than theirs and is therefore more irritating than helpful. The opportunities she gets, in large part because she's seen as white, unfairly hurt other people, and she has to sit with that. It's a thoughtful and uncomfortable look at how difficult it is for a white person to live with discomfort they can't fix and to not make it worse by trying to wave it away or point out their own problems.

That was the positive side of this plot, although I'm still a bit wary and would like to read a review by a Black reviewer to see how well this plot works from their perspective. There are some other choices that I thought landed oddly. One is that the most racist crew member, the one who sparks the most direct conflict with the Black members of the international crew, is a white man from South Africa, which I thought let the United States off the hook too much and externalized the racism a bit too neatly. Another is that the three ships of the expedition are the Niña, the Pinta, and the Santa Maria, and no one in the book comments on this. Given the thoughtful racial themes of the book, I can't imagine this is an accident, and it is in character for United States of this novel to pick those names, but it was an odd intrusion of an unremarked colonial symbol. This may be part of Kowal's attempt to show that Elma is embedded in a racist and sexist world, has limited room to maneuver, and can't solve most of the problems, which is certainly a theme of the series. But it left me unsettled on whether this book was up to fully handling the fraught themes Kowal is invoking.

The other part of the book I found a bit frustrating is that it never seriously engaged with the political argument against Mars colonization, instead treating most of the opponents of space travel as either deluded conspiracy believers or cynical villains. Science fiction is still arguing with William Proxmire even though he's been dead for fifteen years and out of office for thirty. The strong argument against a Mars colony in Elma's world is not funding priorities; it's that even if it's successful, only a tiny fraction of well-connected elites will escape the planet to Mars. This argument is made in the book and Elma dismisses it as a risk she's trying to prevent, but it is correct. There is no conceivable technological future that leads to evacuating the Earth to Mars, but The Fated Sky declines to grapple with the implications of that fact.

There's more that I haven't remarked on, including an ongoing excellent portrayal of the complicated and loving relationship between Elma and her husband, and a surprising development in her antagonistic semi-friendship with the sexist test pilot who becomes the mission captain. I liked how Kowal balanced technical problems with social problems on the long Mars flight; both are serious concerns and they interact with each other in complicated ways.

The details of the perils and joys of manned space flight are excellent, at least so far as I can tell without having done the research that Kowal did. If you want a fictionalized Apollo 13 with higher stakes and less ground support, look no further; this is engrossing stuff. The interpersonal politics and sociology were also fascinating and gripping, but unsettling, in both good ways and bad. I like the challenge that Kowal presents to a white reader, although I'm not sure she was completely in control of it.

Cautiously recommended, although be aware that you'll need to grapple with a sexist and racist society while reading it. Also a content note for somewhat graphic gastrointestinal problems.

Followed by The Relentless Moon.

Rating: 8 out of 10

21 February, 2021 04:43AM

February 20, 2021

hackergotchi for David Bremner

David Bremner

Reading MPS files with glpk

Recently I was asked how to read mps (old school linear programming input) files. I couldn't think of a completely off the shelf way to do, so I write a simple c program to use the glpk library.

Of course in general you would want to do something other than print it out again.

20 February, 2021 05:54PM

Tangling multiple files

I have lately been using org-mode literate programming to generate example code and beamer slides from the same source. I hit a wall trying to re-use functions in multiple files, so I came up with the following hack. Thanks 'ngz' on #emacs and Charles Berry on the org-mode list for suggestions and discussion.

(defun db-extract-tangle-includes ()
  (goto-char (point-min))
  (let ((case-fold-search t)
        (retval nil))
    (while (re-search-forward "^#[+]TANGLE_INCLUDE:" nil t)
      (let ((element (org-element-at-point)))
        (when (eq (org-element-type element) 'keyword)
          (push (org-element-property :value element) retval))))

(defun db-ob-tangle-hook ()
  (let ((includes (db-extract-tangle-includes)))
    (mapc #'org-babel-lob-ingest includes)))

(add-hook 'org-babel-pre-tangle-hook #'db-ob-tangle-hook t)

Use involves something like the following in your org-file.

#+SETUPFILE: presentation-settings.org
#+SETUPFILE: tangle-settings.org
#+TANGLE_INCLUDE: lecture21.org
#+TITLE: GC V: Mark & Sweep with free list

For batch export with make, I do something like

%.tangle-stamp: %.org
    emacs --batch --quick  -l org  -l ${HOME}/.emacs.d/org-settings.el --eval "(org-babel-tangle-file \"$<\")"
    touch $@

20 February, 2021 05:54PM

Yet another buildinfo database.


I previously posted about my extremely quick-and-dirty buildinfo database using buildinfo-sqlite. This year at DebConf, I re-implimented this using PostgreSQL backend, added into some new features.

There is already buildinfo and buildinfos. I was informed I need to think up a name that clearly distinguishes from those two. Thus I give you builtin-pho.

There's a README for how to set up a local database. You'll need 12GB of disk space for the buildinfo files and another 4GB for the database (pro tip: you might want to change the location of your PostgreSQL data_directory, depending on how roomy your /var is)

Demo 1: find things build against old / buggy Build-Depends

select distinct p.source,p.version,d.version, b.path
      binary_packages p, builds b, depends d
      p.suite='sid' and b.source=p.source and
      b.arch_all and p.arch = 'all'
      and p.version = b.version
      and d.id=b.id and d.depend='dh-elpa'
      and d.version < debversion '1.16'

Demo 2: find packages in sid without buildinfo files

select distinct p.source,p.version
      binary_packages p
        select p.source,p.version
from binary_packages p, builds b
      and p.version=b.version
      and ( (b.arch_all and p.arch='all') or
            (b.arch_amd64 and p.arch='amd64') )


Work in progress by an SQL newbie.

20 February, 2021 05:54PM

Dear UNB: please leave my email alone.

1 Background

Apparently motivated by recent phishing attacks against @unb.ca addresses, UNB's Integrated Technology Services unit (ITS) recently started adding banners to the body of email messages. Despite (cough) several requests, they have been unable and/or unwilling to let people opt out of this. Recently ITS has reduced the size of banner; this does not change the substance of what is discussed here. In this blog post I'll try to document some of the reasons this reduces the utility of my UNB email account.

2 What do I know about email?

I have been using email since 1985 1. I have administered my own UNIX-like systems since the mid 1990s. I am a Debian Developer 2. Debian is a mid-sized organization (there are more Debian Developers than UNB faculty members) that functions mainly via email (including discussions and a bug tracker). I maintain a mail user agent (informally, an email client) called notmuch 3. I administer my own (non-UNB) email server. I have spent many hours reading RFCs 4. In summary, my perspective might be different than an enterprise email adminstrator, but I do know something about the nuts and bolts of how email works.

3 What's wrong with a helpful message?

3.1 It's a banner ad.

I don't browse the web without an ad-blocker and I don't watch TV with advertising in it. Apparently the main source of advertising in my life is a service provided by my employer. Some readers will probably dispute my description of a warning label inserted by an email provider as "advertising". Note that is information inserted by a third party to promote their own (well intentioned) agenda, and inserted in an intentionally attention grabbing way. Advertisements from charities are still advertisements. Preventing phishing attacks is important, but so are an almost countless number of priorities of other units of the University. For better or worse those units are not so far able to insert messages into my email. As a thought experiment, imagine inserting a banner into every PDF file stored on UNB servers reminding people of the fiscal year end.

3.2 It makes us look unprofessional.

Because the banner is contained in the body of email messages, it almost inevitably ends up in replies. This lets funding agencies, industrial partners, and potential graduate students know that we consider them as potentially hostile entities. Suggesting that people should edit their replies is not really an acceptable answer, since it suggests that it is acceptable to download the work of maintaining the previous level of functionality onto each user of the system.

3.3 It doesn't help me

I have an archive of 61270 email messages received since 2003. Of these 26215 claim to be from a unb.ca address 5. So historically about 42% of the mail to arrive at my UNB mailbox is internal 6. This means that warnings will occur in the majority of messages I receive. I think the onus is on the proposer to show that a warning that occurs in the large majority of messages will have any useful effect.

3.4 It disrupts my collaboration with open-source projects

Part of my job is to collaborate with various open source projects. A prominent example is Eclipse OMR 7, the technological driver for a collaboration with IBM that has brought millions of dollars of graduate student funding to UNB. Git is now the dominant version control system for open source projects, and one popular way of using git is via git-send-email 8

Adding a banner breaks the delivery of patches by this method. In the a previous experiment I did about a month ago, it "only" caused the banner to end up in the git commit message. Those of you familiar with software developement will know that this is roughly the equivalent of walking out of the bathroom with toilet paper stuck to your shoe. You'd rather avoid it, but it's not fatal. The current implementation breaks things completely by quoted-printable re-encoding the message. In particular '=' gets transformed to '=3D' like the following

-+    gunichar *decoded=g_utf8_to_ucs4_fast (utf8_str, -1, NULL);
-+    const gunichar *p = decoded;
++    gunichar *decoded=3Dg_utf8_to_ucs4_fast (utf8_str, -1, NULL);

I'm not currently sure if this is a bug in git or some kind of failure in the re-encoding. It would likely require an investment of several hours of time to localize that.

3.5 It interferes with the use of cryptography.

Unlike many people, I don't generally read my email on a phone. This means that I don't rely on the previews that are apparently disrupted by the presence of a warning banner. On the other hand I do send and receive OpenPGP signed and encrypted messages. The effects of the banner on both signed and encrypted messages is similar, so I'll stick to discussing signed messages here. There are two main ways of signing a message. The older method, still unfortunately required for some situations is called "inline PGP". The signed region is re-encoded, which causes gpg to issue a warning about a buggy MTA 9, namely gpg: quoted printable character in armor - probably a buggy MTA has been used. This is not exactly confidence inspiring. The more robust and modern standard is PGP/MIME. Here the insertion of a banner does not provoke warnings from the cryptography software, but it does make it much harder to read the message (before and after screenshots are given below). Perhaps more importantly it changes the message from one which is entirely signed or encrypted 10, to one which is partially signed or encrypted. Such messages were instrumental in the EFAIL exploit 11 and will probably soon be rejected by modern email clients.


Figure 1: Intended view of PGP/MIME signed message


Figure 2: View with added banner



On Multics, when I was a high school student


IETF Requests for Comments, which define most of the standards used by email systems.


possibly overcounting some spam as UNB originating email


In case it's not obvious dear reader, communicating with the world outside UNB is part of my job.


Some important projects function exclusively that way. See https://git-send-email.io/ for more information.


Mail Transfer Agent

Author: David Bremner

Created: 2019-05-22 Wed 17:04


20 February, 2021 05:54PM

February 19, 2021

hackergotchi for Jonathan Dowland

Jonathan Dowland

Wrist Watches

red strap

red strap

This is everything I have to say about watches (or time pieces, or chronometers, if you prefer: I don't).

I've always worn a watch, and still do; but I've never really understood the appeal of the kind of luxury watches you see advertise here there and everywhere, with their chunky cases, over-complicated faces and enormous price-tags. So the world of watch-appreciation was closed to me, until my 30th birthday (a while ago) when my wife bought me a Mondaine Evo "Big Date" quartz watch.

It's not an analogue watch nor an "heirloom timepiece", neither of which are properties that matter to me. The large face has almost nothing extraneous on it, although my model includes day-of-the-month. I like it very much.

And so I cracked open the door a little onto the world of watches and watch fashion and had a short spell of interest in some other styles, types, and the like. This drew to a close with buying a selection of cheap, coloured nylon fabric "nato"-style straps. Now whenever I feel the itch for a change, I just change the strap.

Smart Watches have never appealed to me. I can see some of their advantages, but the last thing I need is another gadget to regularly charge, or another avenue to check my email.

I appreciate that wearing a wrist watch at all is anachronistic (sorry), and I did wonder whether it's a habit I could get out of. A few weeks ago, during our endless Lockdown, my watch battery ran out, so I spent a couple of weeks un-learning my reliance on a wristwatch to orient myself. I've managed to get it replaced now (some watch repair places being considered Essential Services) and I'm comfortably back in my default mode of wearing and relying upon it.

19 February, 2021 04:50PM

hackergotchi for Steinar H. Gunderson

Steinar H. Gunderson

plocate LWN post

My debian-devel thread about getting plocate in standard didn't turn into anything in Debian, but evidently, it turned into an LWN post!

My favorite quote from the comments: “It's funny that some people argue that updatedb is too costly while others argue that "find /" (which costs hardly less) is fast enough.”

19 February, 2021 04:49PM

February 18, 2021

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

td 0.0.2 on CRAN: Updated and Expanded

The still very recent td package for accessing the twelvedata API for financial data has been updated and is now at version 0.0.2.

The time_series access point is now vectorised: supply a vector of symbols, and you receive list of data.frame (or xts) objects. See this tweet teasing out the earliest support for this new featire, and showing a quick four-securities plot. We also added simpler accessors get_quote() and get_price() rounding out the basic API support.

One first bug report alerting us to the fact that our use of RcppSimdJson requires an additional sanitizing of the temporary filename if used on Windows. We will fix that properly soon in new release 0.1.5 of that package; in the meantime you can get hot-fix binary for Windows via install.packages("RcppSimdJson", repos="https://ghrr.github.io/drat") from the ghrr drat.

The NEWS entry follows.

Changes in version 0.0.2 (2021-02-18)

  • The time_series is now vectorised and can return a list of return objects when given a vector of symbols

  • The use of tools::R_user_dir() is now conditional on having R 4.0.0 or later, older versions can use env.var for api key

  • New helper function store_key to save api key.

  • New simple accessors get_quote and get_price

Courtesy of my CRANberries, there is a comparison to the previous release. For questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

18 February, 2021 11:23PM

hackergotchi for Daniel Pocock

Daniel Pocock

Influential women

When people ask me about success engaging women in some of the mentoring programs for free, open source software, I never feel comfortable taking credit for that. I feel that it comes down to one simple thing: collaborating with a number of successful and influential women in a variety of different places. Today is the tenth anniversary of the passing of Sally Shaw. Sally had made monumental contributions to the success of the Yarra Yarra Rowing Club (YYRC), even while fighting cancer, raising a family and managing projects for IBM.

Around the same time I met Sally, I had also taken on one of my first web hosting clients, a newly elected politician in the opposition party, Lynne Kosky. Both the YYRC web site and Lynne's web site were among the first projects in my new content management system (CMS), hosted in a GNU/Linux environment. Compared to Sally, I had far fewer opportunities to meet Lynne, her party was elected into government and she became incredibly busy.

If Lynne were alive today, looking at the way her career progressed, there is every chance she would be the state premier, leading the state's response to the pandemic. Moreover, I suspect that if you swapped these two women, they could blaze a trail in each other's workplace just as easily as the career they had chosen. Sadly, both of their lives were cut short for similar reasons.

The 2010/2011 YYRC Annual Report has been used to recognize Sally's contributions:

Sally Shaw sadly lost her battle with Ovarian Cancer in --- 2011. Sally was a significant contributor to Yarra most notably through her involvement with the negotiations with Carey and the subsequent project management of the current Club House. Sally welcomed new Members to the Club through her involvement with novice coaching and later moved into a role on the Committee serving as Secretary and later as Vice President. Sally was a successful oarswoman, winning the Stokes Salver women's trophy in 2002 during the Winter Sculling Series in addition to regatta wins over a decade of rowing. Sally was a Life Member of Yarra and has a racing shell named in her honour.

Sally was committed to living life fully, despite the challenges of regular hospital visits over the last four years and this provided inspiration for her many rowing and other friends. Sally is survived by her parlner Bruce Ricketts, former President of Yarra Yarra, and their two children Grace and Felix, to whom we extend our love and support.

It is interesting to note that Sally is comemorated on the same page as Hubert Frederico, former president of Rowing Victoria, an organization that has produced numerous Olympic and World champions.

In fact, as I went looking through my archive, it wasn't long before I found Sally in a crew with three-time World Champion and Olympian Jane Robinson.

Sally Shaw, Jane Robinson, mixed eight, Yarra Yarra Rowing Club Sally Shaw, Jane Robinson, mixed eight, Yarra Yarra Rowing Club

The same annual report includes a tremendous list of projects ticked off from the Club's Long Term Plan, most notably, the recently completed Club House. It is incredible to see Sally's impact in so many areas of this document. It is even more incredible to think that she was ticking these things off while fighting cancer.

Melbourne's rowing precinct is at a point where the park meets the city center, on the opposite side of a bridge from the main railway station. With major events taking place in Melbourne throughout the year, there are an incredible number of stakeholders and influences on any construction project in this region. Projects like this test the team's skills in every way from compliance to diplomacy.

YYRC, old club house, boathouse drive YYRC, new club house

A video was made in the old Club House before it was demolished. Sally's tells us about her most memorable moment, in the world of free software it sounds a lot like a Code of Conduct violation.

It looks like my camera captured that too:

YYRC, Sally Shaw, swimming YYRC, Sally Shaw, winning medals

Coxing Head of the Yarra

Sally competed both as a rower and a cox. The Head of the Yarra is a gruelling race where approximately one hundred crews row 8.6km upstream:

Head of the Yarra course

It was originally held in late January, the peak of the Australian summer but they now hold it in November to reduce the number of deaths.

It is particularly challenging for the cox because it is not a straight course, in fact, it is every cox's nightmare. This photo captures Sally skillfully steering a combined Yarra/Richmond crew through the treacherous "Big bend", crushing the hopes of another crew as they run into the bank:

YYRC, Head of the Yarra

and then passing a junior crew as they come out of the corner:

YYRC, Head of the Yarra

Meanwhile, Archive.org has captured the original web site we created for Lynne Kosky. Lynne resigned from public office shortly before Sally passed away and Lynne's battle with cancer only become public in 2011. Like Sally, Lynne had ticked off a huge list of projects: Minister for Finance, Minister for Education and the poisoned chalice, Minister for Public Transport.

Lynne Kosky's first web site

Both of these women contributed greatly to the community and to projects that are prominently visible in the city of Melbourne today. Yet their greatest legacy may be the impact they have had on people around them and how we see the potential of women in Australia.

One of the mentors I've worked with called me one day to ask about a female intern spending too much time on political pursuits. I assured him I've seen this before. Despite distractions, or maybe because of them, the intern completed more work than many other interns, male or female.

On the tenth anniversary of Sally's passing, there are news reports about the status of women in other countries, such as Japan, where women have been granted permission to watch men making decisions for them. When I hear about people who claim to represent free software mistreating female employees on sick leave, I imagine them inflicting pain on women like Sally or Lynne. Having a point of reference like this makes it easier to empathize with the victims in those cases.

If you want to comemorate these women or any other victims of cancer, please do not throw IBM employees into the river. A good idea is to simply ask some of the women you work with for suggestions. If you are in Melbourne, you can hire the YYRC Club House for an event or join a Learn to Row program.

Felix, Grace, Sally Shaw, YYRC

18 February, 2021 10:50PM

Julian Andres Klode

APT 2.2 released

APT 2.2.0 marks the freeze of the 2.1 development series and the start of the 2.2 stable series.

Let’s have a look at what changed compared to 2.2. Many of you who run Debian testing or unstable, or Ubuntu groovy or hirsute will already have seen most of those changes.

New features

  • Various patterns related to dependencies, such as ?depends are now available (2.1.16)
  • The Protected field is now supported. It replaces the previous Important field and is like Essential, but only for installed packages (some minor more differences maybe in terms of ordering the installs).
  • The update command has gained an --error-on=any option that makes it error out on any failure, not just what it considers persistent ons.
  • The rred method can now be used as a standalone program to merge pdiff files
  • APT now implements phased updates. Phasing is used in Ubuntu to slow down and control the roll out of updates in the -updates pocket, but has previously only been available to desktop users using update-manager.

Other behavioral changes

  • The kernel autoremoval helper code has been rewritten from shell in C++ and now runs at run-time, rather than at kernel install time, in order to correctly protect the kernel that is running now, rather than the kernel that was running when we were installing the newest one.

    It also now protects only up to 3 kernels, instead of up to 4, as was originally intended, and was the case before 1.1 series. This avoids /boot partitions from running out of space, especially on Ubuntu which has boot partitions sized for the original spec.

Performance improvements

  • The cache is now hashed using XXH3 instead of Adler32 (or CRC32c on SSE4.2 platforms)
  • The hash table size has been increased

Bug fixes

  • * wildcards work normally again (since 2.1.0)
  • The cache file now includes all translation files in /var/lib/apt/lists, so multi-user systems with different locales correctly show translated descriptions now.
  • URLs are no longer dequoted on redirects only to be requoted again, fixing some redirects where servers did not expect different quoting.
  • Immediate configuration is now best-effort, and failure is no longer fatal.
  • various changes to solver marking leading to different/better results in some cases (since 2.1.0)
  • The lower level I/O bits of the HTTP method have been rewritten to hopefully improve stability
  • The HTTP method no longer infinitely retries downloads on some connection errors
  • The pkgnames command no longer accidentally includes source packages
  • Various fixes from fuzzing efforts by David

Security fixes

  • Out-of-bound reads in ar and tar implementations (CVE-2020-3810, 2.1.2)
  • Integer overflows in ar and tar (CVE-2020-27350, 2.1.13)

(all of which have been backported to all stable series, back all the way to* series in jessie eLTS)


  • N/A - there were no breaking changes in apt 2.2 that we are aware of.


  • apt-key(1) is scheduled to be removed for Q2/2022, and several new warnings have been added.

    apt-key was made obsolete in version, released in January 2010, by /etc/apt/trusted.gpg.d becoming a supported place to drop additional keyring files, and was since then only intended for deleting keys in the legacy trusted.gpg keyring.

    Please manage files in trusted.gpg.d yourself; or place them in a different location such as /etc/apt/keyrings (or make up your own, there’s no standard location) or /usr/share/keyrings, and use signed-by in the sources.list.d files.

    The legacy trusted.gpg keyring still works, but will also stop working eventually. Please make sure you have all your keys in trusted.gpg.d. Warnings might be added in the upcoming months when a signature could not be verified using just trusted.gpg.d.

    Future versions of APT might switch away from GPG.

  • As a reminder, regular expressions and wildcards other than * inside package names are deprecated (since 2.0). They are not available anymore in apt(8), and will be removed for safety reasons in apt-get in a later release.

18 February, 2021 08:09PM

hackergotchi for Jonathan McDowell

Jonathan McDowell

Hacking and Bricking the EE Opsrey 2 Mini

I’ve mentioned in the past my twisted EE network setup from when I moved in to my current house. The 4GEE WiFi Mini (also known as the EE Osprey 2 Mini or the EE40VB, and actually a rebadged Alcatel Y853VB) has been sitting unused since then, so I figured I’d see about trying to get a shell on it.

TL;DR: Of course it’s running Linux, there’s a couple of test points internally which bring out the serial console, but after finding those and logging in I discovered it’s running ADB on port 5555 quite happily available without authentication both via wifi and the USB port. So if you have physical or local network access, instant root shell. Well done, folks. And then I bricked it before I could do anything more interesting.

There’s a lack of information about this device out there - most of the links I can find are around removing the SIM lock - so I thought I’d document the pieces I found just in case anyone else is trying to figure it out. It’s based around a Qualcomm MDM9607 SoC, paired with 64M RAM and 256M NAND flash. Wifi is via an RTL8192ES. Kernel is 3.18.20. Busybox is v1.23.1. It’s running dnsmasq but I didn’t grab the version. Of course there’s no source or offer of source provided.

Taking it apart is fairly easy. There’s a single screw to remove, just beside the SIM slot. The coloured rim can then be carefully pried away from the back, revealing the battery. There are then 4 screws in the corners which need removed in order to be able to lift out the actual PCB and gain access to the serial console test points.

EE40VB PCB serial console test points

My mistake was going poking around trying to figure out where the updates are downloaded from - I know I’m running a slightly older release than what’s current, and the device can do an automatic download + update. Top tip; don’t run Jrdrecovery. It’ll error on finding /cache/update.zip and wipe the main partition anyway. That’ll leave you in a boot loop where the device boots the recovery partition which tries to install /cache/update.zip which of course still doesn’t exist.

So. Where next? First, I need to get the device into a state where I can actually do something other than watch it boot into recovery, fail to flash and reboot. Best guess at present is to try and get it to enter the Qualcomm EDL (Emergency Download) mode. That might be possible with a custom USB cable that grounds D+ on boot. Alternatively I need to probe some of the other test points on the PCB and see if grounding any of those helps enter EDL mode. I then need a suitable “firehose” OEM-signed programmer image. And then I need to actually get hold of a proper EE40VB firmware image, either via one of the OTA update files or possibly via an Alcatel ADSU image (though no idea how to get hold of one, other than by posting to a random GSM device forum and hoping for the kindness of strangers). More updates if/when I make progress…

Qualcomm bootloader log
Format: Log Type - Time(microsec) - Message - Optional Info
Log Type: B - Since Boot(Power On Reset),  D - Delta,  S - Statistic
S - Boot Config, 0x000002e1
B -    105194 - SBL1, Start
D -     61885 - QSEE Image Loaded, Delta - (451964 Bytes)
D -     30286 - RPM Image Loaded, Delta - (151152 Bytes)
B -    459330 - Roger:boot_jrd_oem_main
B -    461526 - Welcome to key_check_poweron!!!
B -    466436 - REG0x00, rc=47
B -    469120 - REG0x01, rc=1f
B -    472018 - REG0x02, rc=1c
B -    474885 - REG0x03, rc=47
B -    477782 - REG0x04, rc=b2
B -    480558 - REG0x05, rc=
B -    483272 - REG0x06, rc=9e
B -    486139 - REG0x07, rc=
B -    488854 - REG0x08, rc=a4
B -    491721 - REG0x09, rc=80
B -    494130 - bq24295_probe: vflt/vsys/vprechg=0mV/0mV/0mV, tprechg/tfastchg=0Min/0Min, [0C, 0C]
B -    511546 - come to calculate vol and temperature!!
B -    511637 - ##############battery_core_convert_vntc: NTC_voltage=1785690
B -    517280 - battery_core_convert_vntc: <-44C, 1785690uV>, present=0
B -    529358 - bq24295_set_current_limit: setting=0mA, mode=-1, input/fastchg/prechg/termchg=-1mA/0mA/0mA/0mA
B -    534360 - bq24295_set_charge_current, rc=0,reg_val=0,i=0
B -    539636 - bq24295_enable_charge: setting=0, chg_enable=-1, otg_enable=0
B -    546072 - bq24295_enable_charging: enable_charging=0
B -    552172 - bq24295_set_current_limit: setting=0mA, mode=-1, input/fastchg/prechg/termchg=-1mA/0mA/0mA/0mA
B -    561566 - bq24295_set_charge_current, rc=0,reg_val=0,i=0
B -    567056 - bq24295_enable_charge: setting=0, chg_enable=0, otg_enable=0
B -    579286 - come to calculate vol and temperature!!
B -    579378 - ##############battery_core_convert_vntc: NTC_voltage=1785777
B -    585539 - battery_core_convert_vntc: <-44C, 1785777uV>, present=0
B -    597617 - charge_main: battery is plugout!!
B -    597678 - Welcome to pca955x_probe!!!
B -    601063 - pca955x_probe: PCA955X probed successfully!
D -     27511 - APPSBL Image Loaded, Delta - (179348 Bytes)
B -    633271 - QSEE Execution, Start
D -       213 - QSEE Execution, Delta
B -    638944 - >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>Start writting JRD RECOVERY BOOT
B -    650107 - >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>Start writting  RECOVERY BOOT
B -    653218 - >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>read_buf[0] == 0
B -    659044 - SBL1, End
D -    556137 - SBL1, Delta
S - Throughput, 2000 KB/s  (782884 Bytes,  278155 us)
S - DDR Frequency, 240 MHz
littlekernel aboot log
Android Bootloader - UART_DM Initialized!!!
[0] welcome to lk

[0] SCM call: 0x2000601 failed with :fffffffc
[0] Failed to initialize SCM
[10] platform_init()
[10] target_init()
[10] smem ptable found: ver: 4 len: 17
[10] ERROR: No devinfo partition found
[10] Neither 'config' nor 'frp' partition found
[30] voltage of NTC  is 1789872!
[30] voltage of BAT  is 3179553!
[30] usb present is 1!
[30] Loading (boot) image (4171776): start
[530] Loading (boot) image (4171776): done
[540] DTB Total entry: 25, DTB version: 3
[540] Using DTB entry 0x00000129/00010000/0x00000008/0 for device 0x00000129/00010000/0x00010008/0
[560] come to jrd_target_pause_for_battery_charge!
[570] power_on_status.hard_reset = 0x0
[570] power_on_status.smpl = 0x0
[570] power_on_status.rtc = 0x0
[580] power_on_status.dc_chg = 0x0
[580] power_on_status.usb_chg = 0x0
[580] power_on_status.pon1 = 0x1
[590] power_on_status.cblpwr = 0x0
[590] power_on_status.kpdpwr = 0x0
[590] power_on_status.bugflag = 0x0
[590] cmdline: noinitrd  rw console=ttyHSL0,115200,n8 androidboot.hardware=qcom ehci-hcd.park=3 msm_rtb.filter=0x37 lpm_levels.sleep_disabled=1  earlycon=msm_hsl_uart,0x78b3000  androidboot.serialno=7e6ba58c androidboot.baseband=msm rootfstype=ubifs rootflags=b
[620] Updating device tree: start
[720] Updating device tree: done
[720] booting linux @ 0x80008000, ramdisk @ 0x80008000 (0), tags/device tree @ 0x81e00000
Linux kernel console boot log
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 3.18.20 (linux3@linux3) (gcc version 4.9.2 (GCC) ) #1 PREEMPT Thu Aug 10 11:57:07 CST 2017
[    0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c53c7d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine model: Qualcomm Technologies, Inc. MDM 9607 MTP
[    0.000000] Early serial console at I/O port 0x0 (options '')
[    0.000000] bootconsole [uart0] enabled
[    0.000000] Reserved memory: reserved region for node 'modem_adsp_region@0': base 0x82a00000, size 56 MiB
[    0.000000] Reserved memory: reserved region for node 'external_image_region@0': base 0x87c00000, size 4 MiB
[    0.000000] Removed memory: created DMA memory pool at 0x82a00000, size 56 MiB
[    0.000000] Reserved memory: initialized node modem_adsp_region@0, compatible id removed-dma-pool
[    0.000000] Removed memory: created DMA memory pool at 0x87c00000, size 4 MiB
[    0.000000] Reserved memory: initialized node external_image_region@0, compatible id removed-dma-pool
[    0.000000] cma: Reserved 4 MiB at 0x87800000
[    0.000000] Memory policy: Data cache writeback
[    0.000000] CPU: All CPU(s) started in SVC mode.
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 17152
[    0.000000] Kernel command line: noinitrd  rw console=ttyHSL0,115200,n8 androidboot.hardware=qcom ehci-hcd.park=3 msm_rtb.filter=0x37 lpm_levels.sleep_disabled=1  earlycon=msm_hsl_uart,0x78b3000  androidboot.serialno=7e6ba58c androidboot.baseband=msm rootfstype=ubifs rootflags=bulk_read root=ubi0:rootfs ubi.mtd=16
[    0.000000] PID hash table entries: 512 (order: -1, 2048 bytes)
[    0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
[    0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
[    0.000000] Memory: 54792K/69632K available (5830K kernel code, 399K rwdata, 2228K rodata, 276K init, 830K bss, 14840K reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xc8800000 - 0xff000000   ( 872 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xc8000000   ( 128 MB)
[    0.000000]     modules : 0xbf000000 - 0xc0000000   (  16 MB)
[    0.000000]       .text : 0xc0008000 - 0xc07e6c38   (8060 kB)
[    0.000000]       .init : 0xc07e7000 - 0xc082c000   ( 276 kB)
[    0.000000]       .data : 0xc082c000 - 0xc088fdc0   ( 400 kB)
[    0.000000]        .bss : 0xc088fe84 - 0xc095f798   ( 831 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] NR_IRQS:16 nr_irqs:16 16
[    0.000000] GIC CPU mask not found - kernel will fail to boot.
[    0.000000] GIC CPU mask not found - kernel will fail to boot.
[    0.000000] mpm_init_irq_domain(): Cannot find irq controller for qcom,gpio-parent
[    0.000000] MPM 1 irq mapping errored -517
[    0.000000] Architected mmio timer(s) running at 19.20MHz (virt).
[    0.000011] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 3579139424256ns
[    0.007975] Switching to timer-based delay loop, resolution 52ns
[    0.013969] Switched to clocksource arch_mem_counter
[    0.019687] Console: colour dummy device 80x30
[    0.023344] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=192000)
[    0.033666] pid_max: default: 32768 minimum: 301
[    0.038411] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.044902] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.052445] CPU: Testing write buffer coherency: ok
[    0.057057] Setting up static identity map for 0x8058aac8 - 0x8058ab20
[    0.064242]
[    0.064242] **********************************************************
[    0.077817] **                                                      **
[    0.084302] ** trace_printk() being used. Allocating extra memory.  **
[    0.090781] **                                                      **
[    0.097320] ** This means that this is a DEBUG kernel and it is     **
[    0.103802] ** unsafe for produciton use.                           **
[    0.110339] **                                                      **
[    0.116850] ** If you see this message and you are not debugging    **
[    0.123333] ** the kernel, report this immediately to your vendor!  **
[    0.129870] **                                                      **
[    0.142865] **********************************************************
[    0.150225] MSM Memory Dump base table set up
[    0.153739] MSM Memory Dump apps data table set up
[    0.168125] VFP support v0.3: implementor 41 architecture 2 part 30 variant 7 rev 5
[    0.176332] pinctrl core: initialized pinctrl subsystem
[    0.180930] regulator-dummy: no parameters
[    0.215338] NET: Registered protocol family 16
[    0.220475] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.284034] cpuidle: using governor ladder
[    0.314026] cpuidle: using governor menu
[    0.344024] cpuidle: using governor qcom
[    0.355452] msm_watchdog b017000.qcom,wdt: wdog absent resource not present
[    0.361656] msm_watchdog b017000.qcom,wdt: MSM Watchdog Initialized
[    0.371373] irq: no irq domain found for /soc/pinctrl@1000000 !
[    0.381268] spmi_pmic_arb 200f000.qcom,spmi: PMIC Arb Version-2 0x20010000
[    0.389733] platform 4080000.qcom,mss: assigned reserved memory node modem_adsp_region@0
[    0.397409] mem_acc_corner: 0 <--> 0 mV
[    0.401937] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
[    0.408966] hw-breakpoint: maximum watchpoint size is 8 bytes.
[    0.416287] __of_mpm_init(): MPM driver mapping exists
[    0.420940] msm_rpm_glink_dt_parse: qcom,rpm-glink compatible not matches
[    0.427235] msm_rpm_dev_probe: APSS-RPM communication over SMD
[    0.432977] smd_open() before smd_init()
[    0.437544] msm_mpm_dev_probe(): Cannot get clk resource for XO: -517
[    0.445730] smd_channel_probe_now: allocation table not initialized
[    0.453100] mdm9607_s1: 1050 <--> 1350 mV at 1225 mV normal idle
[    0.458566] spm_regulator_probe: name=mdm9607_s1, range=LV, voltage=1225000 uV, mode=AUTO, step rate=4800 uV/us
[    0.468817] cpr_efuse_init: apc_corner: efuse_addr = 0x000a4000 (len=0x1000)
[    0.475353] cpr_read_fuse_revision: apc_corner: fuse revision = 2
[    0.481345] cpr_parse_speed_bin_fuse: apc_corner: [row: 37]: 0x79e8bd327e6ba58c, speed_bits = 4
[    0.490124] cpr_pvs_init: apc_corner: pvs voltage: [1050000 1100000 1275000] uV
[    0.497342] cpr_pvs_init: apc_corner: ceiling voltage: [1050000 1225000 1350000] uV
[    0.504979] cpr_pvs_init: apc_corner: floor voltage: [1050000 1050000 1150000] uV
[    0.513125] i2c-msm-v2 78b8000.i2c: probing driver i2c-msm-v2
[    0.518335] i2c-msm-v2 78b8000.i2c: error on clk_get(core_clk):-517
[    0.524478] i2c-msm-v2 78b8000.i2c: error probe() failed with err:-517
[    0.531111] i2c-msm-v2 78b7000.i2c: probing driver i2c-msm-v2
[    0.536788] i2c-msm-v2 78b7000.i2c: error on clk_get(core_clk):-517
[    0.542886] i2c-msm-v2 78b7000.i2c: error probe() failed with err:-517
[    0.549618] i2c-msm-v2 78b9000.i2c: probing driver i2c-msm-v2
[    0.555202] i2c-msm-v2 78b9000.i2c: error on clk_get(core_clk):-517
[    0.561374] i2c-msm-v2 78b9000.i2c: error probe() failed with err:-517
[    0.570613] msm-thermal soc:qcom,msm-thermal: msm_thermal:Failed reading node=/soc/qcom,msm-thermal, key=qcom,core-limit-temp. err=-22. KTM continues
[    0.583049] msm-thermal soc:qcom,msm-thermal: probe_therm_reset:Failed reading node=/soc/qcom,msm-thermal, key=qcom,therm-reset-temp err=-22. KTM continues
[    0.596926] msm_thermal:msm_thermal_dev_probe Failed reading node=/soc/qcom,msm-thermal, key=qcom,online-hotplug-core. err:-517
[    0.609370] sps:sps is ready.
[    0.613137] msm_rpm_glink_dt_parse: qcom,rpm-glink compatible not matches
[    0.619020] msm_rpm_dev_probe: APSS-RPM communication over SMD
[    0.625773] mdm9607_s2: 750 <--> 1275 mV at 750 mV normal idle
[    0.631584] mdm9607_s3_level: 0 <--> 0 mV at 0 mV normal idle
[    0.637085] mdm9607_s3_level_ao: 0 <--> 0 mV at 0 mV normal idle
[    0.643092] mdm9607_s3_floor_level: 0 <--> 0 mV at 0 mV normal idle
[    0.649512] mdm9607_s3_level_so: 0 <--> 0 mV at 0 mV normal idle
[    0.655750] mdm9607_s4: 1800 <--> 1950 mV at 1800 mV normal idle
[    0.661791] mdm9607_l1: 1250 mV normal idle
[    0.666090] mdm9607_l2: 1800 mV normal idle
[    0.670276] mdm9607_l3: 1800 mV normal idle
[    0.674541] mdm9607_l4: 3075 mV normal idle
[    0.678743] mdm9607_l5: 1700 <--> 3050 mV at 1700 mV normal idle
[    0.684904] mdm9607_l6: 1700 <--> 3050 mV at 1700 mV normal idle
[    0.690892] mdm9607_l7: 1700 <--> 1900 mV at 1700 mV normal idle
[    0.697036] mdm9607_l8: 1800 mV normal idle
[    0.701238] mdm9607_l9: 1200 <--> 1250 mV at 1200 mV normal idle
[    0.707367] mdm9607_l10: 1050 mV normal idle
[    0.711662] mdm9607_l11: 1800 mV normal idle
[    0.716089] mdm9607_l12_level: 0 <--> 0 mV at 0 mV normal idle
[    0.721717] mdm9607_l12_level_ao: 0 <--> 0 mV at 0 mV normal idle
[    0.727946] mdm9607_l12_level_so: 0 <--> 0 mV at 0 mV normal idle
[    0.734099] mdm9607_l12_floor_lebel: 0 <--> 0 mV at 0 mV normal idle
[    0.740706] mdm9607_l13: 1800 <--> 2850 mV at 2850 mV normal idle
[    0.746883] mdm9607_l14: 2650 <--> 3000 mV at 2650 mV normal idle
[    0.752515] msm_mpm_dev_probe(): Cannot get clk resource for XO: -517
[    0.759036] cpr_efuse_init: apc_corner: efuse_addr = 0x000a4000 (len=0x1000)
[    0.765807] cpr_read_fuse_revision: apc_corner: fuse revision = 2
[    0.771809] cpr_parse_speed_bin_fuse: apc_corner: [row: 37]: 0x79e8bd327e6ba58c, speed_bits = 4
[    0.780586] cpr_pvs_init: apc_corner: pvs voltage: [1050000 1100000 1275000] uV
[    0.787808] cpr_pvs_init: apc_corner: ceiling voltage: [1050000 1225000 1350000] uV
[    0.795443] cpr_pvs_init: apc_corner: floor voltage: [1050000 1050000 1150000] uV
[    0.803094] cpr_init_cpr_parameters: apc_corner: up threshold = 2, down threshold = 3
[    0.810752] cpr_init_cpr_parameters: apc_corner: CPR is enabled by default.
[    0.817687] cpr_init_cpr_efuse: apc_corner: [row:65] = 0x15000277277383
[    0.824272] cpr_init_cpr_efuse: apc_corner: CPR disable fuse = 0
[    0.830225] cpr_init_cpr_efuse: apc_corner: Corner[1]: ro_sel = 0, target quot = 631
[    0.837976] cpr_init_cpr_efuse: apc_corner: Corner[2]: ro_sel = 0, target quot = 631
[    0.845703] cpr_init_cpr_efuse: apc_corner: Corner[3]: ro_sel = 0, target quot = 899
[    0.853592] cpr_config: apc_corner: Timer count: 0x17700 (for 5000 us)
[    0.860426] apc_corner: 0 <--> 0 mV
[    0.864044] i2c-msm-v2 78b8000.i2c: probing driver i2c-msm-v2
[    0.869261] i2c-msm-v2 78b8000.i2c: error on clk_get(core_clk):-517
[    0.875492] i2c-msm-v2 78b8000.i2c: error probe() failed with err:-517
[    0.882225] i2c-msm-v2 78b7000.i2c: probing driver i2c-msm-v2
[    0.887775] i2c-msm-v2 78b7000.i2c: error on clk_get(core_clk):-517
[    0.893941] i2c-msm-v2 78b7000.i2c: error probe() failed with err:-517
[    0.900719] i2c-msm-v2 78b9000.i2c: probing driver i2c-msm-v2
[    0.906256] i2c-msm-v2 78b9000.i2c: error on clk_get(core_clk):-517
[    0.912430] i2c-msm-v2 78b9000.i2c: error probe() failed with err:-517
[    0.919472] msm-thermal soc:qcom,msm-thermal: msm_thermal:Failed reading node=/soc/qcom,msm-thermal, key=qcom,core-limit-temp. err=-22. KTM continues
[    0.932372] msm-thermal soc:qcom,msm-thermal: probe_therm_reset:Failed reading node=/soc/qcom,msm-thermal,
key=qcom,therm-reset-temp err=-22. KTM continues
[    0.946361] msm_thermal:get_kernel_cluster_info CPU0 topology not initialized.
[    0.953824] cpu cpu0: dev_pm_opp_get_opp_count: device OPP not found (-19)
[    0.960300] msm_thermal:get_cpu_freq_plan_len Error reading CPU0 freq table len. error:-19
[    0.968533] msm_thermal:vdd_restriction_reg_init Defer vdd rstr freq init.
[    0.975846] cpu cpu0: dev_pm_opp_get_opp_count: device OPP not found (-19)
[    0.982219] msm_thermal:get_cpu_freq_plan_len Error reading CPU0 freq table len. error:-19
[    0.991378] cpu cpu0: dev_pm_opp_get_opp_count: device OPP not found (-19)
[    0.997544] msm_thermal:get_cpu_freq_plan_len Error reading CPU0 freq table len. error:-19
[    1.013642] qcom,gcc-mdm9607 1800000.qcom,gcc: Registered GCC clocks
[    1.019451] clock-a7 b010008.qcom,clock-a7: Speed bin: 4 PVS Version: 0
[    1.025693] a7ssmux: set OPP pair(400000000 Hz: 1 uV) on cpu0
[    1.031314] a7ssmux: set OPP pair(1305600000 Hz: 7 uV) on cpu0
[    1.038805] i2c-msm-v2 78b8000.i2c: probing driver i2c-msm-v2
[    1.043587] AXI: msm_bus_scale_register_client(): msm_bus_scale_register_client: Bus driver not ready.
[    1.052935] i2c-msm-v2 78b8000.i2c: msm_bus_scale_register_client(mstr-id:86):0 (not a problem)
[    1.062006] irq: no irq domain found for /soc/wcd9xxx-irq !
[    1.069884] i2c-msm-v2 78b7000.i2c: probing driver i2c-msm-v2
[    1.074814] AXI: msm_bus_scale_register_client(): msm_bus_scale_register_client: Bus driver not ready.
[    1.083716] i2c-msm-v2 78b7000.i2c: msm_bus_scale_register_client(mstr-id:86):0 (not a problem)
[    1.093850] i2c-msm-v2 78b9000.i2c: probing driver i2c-msm-v2
[    1.098889] AXI: msm_bus_scale_register_client(): msm_bus_scale_register_client: Bus driver not ready.
[    1.107779] i2c-msm-v2 78b9000.i2c: msm_bus_scale_register_client(mstr-id:86):0 (not a problem)
[    1.167871] KPI: Bootloader start count = 24097
[    1.171364] KPI: Bootloader end count = 48481
[    1.175855] KPI: Bootloader display count = 3884474147
[    1.180825] KPI: Bootloader load kernel count = 16420
[    1.185905] KPI: Kernel MPM timestamp = 105728
[    1.190286] KPI: Kernel MPM Clock frequency = 32768
[    1.195209] socinfo_print: v0.10, id=297, ver=1.0, raw_id=72, raw_ver=0, hw_plat=8, hw_plat_ver=65536
[    1.195209]  accessory_chip=0, hw_plat_subtype=0, pmic_model=65539, pmic_die_revision=131074 foundry_id=0 serial_number=2120983948
[    1.216731] sdcard_ext_vreg: no parameters
[    1.220555] rome_vreg: no parameters
[    1.224133] emac_lan_vreg: no parameters
[    1.228177] usbcore: registered new interface driver usbfs
[    1.233156] usbcore: registered new interface driver hub
[    1.238578] usbcore: registered new device driver usb
[    1.244507] cpufreq: driver msm up and running
[    1.248425] ION heap system created
[    1.251895] msm_bus_fabric_init_driver
[    1.262563] qcom,qpnp-power-on qpnp-power-on-c7303800: PMIC@SID0 Power-on reason: Triggered from PON1 (secondary PMIC) and 'cold' boot
[    1.273747] qcom,qpnp-power-on qpnp-power-on-c7303800: PMIC@SID0: Power-off reason: Triggered from UVLO (Under Voltage Lock Out)
[    1.285430] input: qpnp_pon as /devices/virtual/input/input0
[    1.291246] PMIC@SID0: PM8019 v2.2 options: 3, 2, 2, 2
[    1.296706] Advanced Linux Sound Architecture Driver Initialized.
[    1.302493] Add group failed
[    1.305291] cfg80211: Calling CRDA to update world regulatory domain
[    1.311216] cfg80211: World regulatory domain updated:
[    1.317109] Switched to clocksource arch_mem_counter
[    1.334091] cfg80211:  DFS Master region: unset
[    1.337418] cfg80211:   (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp), (dfs_cac_time)
[    1.354087] cfg80211:   (2402000 KHz - 2472000 KHz @ 40000 KHz), (N/A, 2000 mBm), (N/A)
[    1.361055] cfg80211:   (2457000 KHz - 2482000 KHz @ 40000 KHz), (N/A, 2000 mBm), (N/A)
[    1.370545] NET: Registered protocol family 2
[    1.374082] cfg80211:   (2474000 KHz - 2494000 KHz @ 20000 KHz), (N/A, 2000 mBm), (N/A)
[    1.381851] cfg80211:   (5170000 KHz - 5250000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.389876] cfg80211:   (5250000 KHz - 5330000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.397857] cfg80211:   (5490000 KHz - 5710000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.405841] cfg80211:   (5735000 KHz - 5835000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.413795] cfg80211:   (57240000 KHz - 63720000 KHz @ 2160000 KHz), (N/A, 0 mBm), (N/A)
[    1.422355] TCP established hash table entries: 1024 (order: 0, 4096 bytes)
[    1.428921] TCP bind hash table entries: 1024 (order: 0, 4096 bytes)
[    1.435192] TCP: Hash tables configured (established 1024 bind 1024)
[    1.441528] TCP: reno registered
[    1.444738] UDP hash table entries: 256 (order: 0, 4096 bytes)
[    1.450521] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
[    1.456950] NET: Registered protocol family 1
[    1.462779] futex hash table entries: 256 (order: -1, 3072 bytes)
[    1.474555] msgmni has been set to 115
[    1.478551] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[    1.485041] io scheduler noop registered
[    1.488818] io scheduler deadline registered
[    1.493200] io scheduler cfq registered (default)
[    1.502142] msm_rpm_log_probe: OK
[    1.506717] msm_serial_hs module loaded
[    1.509803] msm_serial_hsl_probe: detected port #0 (ttyHSL0)
[    1.515324] AXI: get_pdata(): Error: Client name not found
[    1.520626] AXI: msm_bus_cl_get_pdata(): client has to provide missing entry for successful registration
[    1.530171] msm_serial_hsl_probe: Bus scaling is disabled                      [    1.074814] AXI: msm_bus_scale_register_client(): msm_bus_scale_register_client: Bus driver not ready.
[    1.083716] i2c-msm-v2 78b7000.i2c: msm_bus_scale_register_client(mstr-id:86):0 (not a problem)
[    1.093850] i2c-msm-v2 78b9000.i2c: probing driver i2c-msm-v2
[    1.098889] AXI: msm_bus_scale_register_client(): msm_bus_scale_register_client: Bus driver not ready.
[    1.107779] i2c-msm-v2 78b9000.i2c: msm_bus_scale_register_client(mstr-id:86):0 (not a problem)
[    1.167871] KPI: Bootloader start count = 24097
[    1.171364] KPI: Bootloader end count = 48481
[    1.175855] KPI: Bootloader display count = 3884474147
[    1.180825] KPI: Bootloader load kernel count = 16420
[    1.185905] KPI: Kernel MPM timestamp = 105728
[    1.190286] KPI: Kernel MPM Clock frequency = 32768
[    1.195209] socinfo_print: v0.10, id=297, ver=1.0, raw_id=72, raw_ver=0, hw_plat=8, hw_plat_ver=65536
[    1.195209]  accessory_chip=0, hw_plat_subtype=0, pmic_model=65539, pmic_die_revision=131074 foundry_id=0 serial_number=2120983948
[    1.216731] sdcard_ext_vreg: no parameters
[    1.220555] rome_vreg: no parameters
[    1.224133] emac_lan_vreg: no parameters
[    1.228177] usbcore: registered new interface driver usbfs
[    1.233156] usbcore: registered new interface driver hub
[    1.238578] usbcore: registered new device driver usb
[    1.244507] cpufreq: driver msm up and running
[    1.248425] ION heap system created
[    1.251895] msm_bus_fabric_init_driver
[    1.262563] qcom,qpnp-power-on qpnp-power-on-c7303800: PMIC@SID0 Power-on reason: Triggered from PON1 (secondary PMIC) and 'cold' boot
[    1.273747] qcom,qpnp-power-on qpnp-power-on-c7303800: PMIC@SID0: Power-off reason: Triggered from UVLO (Under Voltage Lock Out)
[    1.285430] input: qpnp_pon as /devices/virtual/input/input0
[    1.291246] PMIC@SID0: PM8019 v2.2 options: 3, 2, 2, 2
[    1.296706] Advanced Linux Sound Architecture Driver Initialized.
[    1.302493] Add group failed
[    1.305291] cfg80211: Calling CRDA to update world regulatory domain
[    1.311216] cfg80211: World regulatory domain updated:
[    1.317109] Switched to clocksource arch_mem_counter
[    1.334091] cfg80211:  DFS Master region: unset
[    1.337418] cfg80211:   (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp), (dfs_cac_time)
[    1.354087] cfg80211:   (2402000 KHz - 2472000 KHz @ 40000 KHz), (N/A, 2000 mBm), (N/A)
[    1.361055] cfg80211:   (2457000 KHz - 2482000 KHz @ 40000 KHz), (N/A, 2000 mBm), (N/A)
[    1.370545] NET: Registered protocol family 2
[    1.374082] cfg80211:   (2474000 KHz - 2494000 KHz @ 20000 KHz), (N/A, 2000 mBm), (N/A)
[    1.381851] cfg80211:   (5170000 KHz - 5250000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.389876] cfg80211:   (5250000 KHz - 5330000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.397857] cfg80211:   (5490000 KHz - 5710000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.405841] cfg80211:   (5735000 KHz - 5835000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    1.413795] cfg80211:   (57240000 KHz - 63720000 KHz @ 2160000 KHz), (N/A, 0 mBm), (N/A)
[    1.422355] TCP established hash table entries: 1024 (order: 0, 4096 bytes)
[    1.428921] TCP bind hash table entries: 1024 (order: 0, 4096 bytes)
[    1.435192] TCP: Hash tables configured (established 1024 bind 1024)
[    1.441528] TCP: reno registered
[    1.444738] UDP hash table entries: 256 (order: 0, 4096 bytes)
[    1.450521] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
[    1.456950] NET: Registered protocol family 1
[    1.462779] futex hash table entries: 256 (order: -1, 3072 bytes)
[    1.474555] msgmni has been set to 115
[    1.478551] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[    1.485041] io scheduler noop registered
[    1.488818] io scheduler deadline registered
[    1.493200] io scheduler cfq registered (default)
[    1.502142] msm_rpm_log_probe: OK
[    1.506717] msm_serial_hs module loaded
[    1.509803] msm_serial_hsl_probe: detected port #0 (ttyHSL0)
[    1.515324] AXI: get_pdata(): Error: Client name not found
[    1.520626] AXI: msm_bus_cl_get_pdata(): client has to provide missing entry for successful registration
[    1.530171] msm_serial_hsl_probe: Bus scaling is disabled
[    1.535696] 78b3000.serial: ttyHSL0 at MMIO 0x78b3000 (irq = 153, base_baud = 460800�[    1.544155] msm_hsl_console_setup: console setup on port #0
[    1.548727] console [ttyHSL0] enabled
[    1.548727] console [ttyHSL0] enabled
[    1.556014] bootconsole [uart0] disabled
[    1.556014] bootconsole [uart0] disabled
[    1.564212] msm_serial_hsl_init: driver initialized
[    1.578450] brd: module loaded
[    1.582920] loop: module loaded
[    1.589183] sps: BAM device 0x07984000 is not registered yet.
[    1.594234] sps:BAM 0x07984000 is registered.
[    1.598072] msm_nand_bam_init: msm_nand_bam_init: BAM device registered: bam_handle 0xc69f6400
[    1.607103] sps:BAM 0x07984000 (va:0xc89a0000) enabled: ver:0x18, number of pipes:7
[    1.616588] msm_nand_parse_smem_ptable: Parsing partition table info from SMEM
[    1.622805] msm_nand_parse_smem_ptable: SMEM partition table found: ver: 4 len: 17
[    1.630391] msm_nand_version_check: nand_major:1, nand_minor:5, qpic_major:1, qpic_minor:5
[    1.638642] msm_nand_scan: NAND Id: 0x1590aa98 Buswidth: 8Bits Density: 256 MByte
[    1.646069] msm_nand_scan: pagesize: 2048 Erasesize: 131072 oobsize: 128 (in Bytes)
[    1.653676] msm_nand_scan: BCH ECC: 8 Bit
[    1.657710] msm_nand_scan: CFG0: 0x290408c0,           CFG1: 0x0804715c
[    1.657710]             RAWCFG0: 0x2b8400c0,        RAWCFG1: 0x0005055d
[    1.657710]           ECCBUFCFG: 0x00000203,      ECCBCHCFG: 0x42040d10
[    1.657710]           RAWECCCFG: 0x42000d11, BAD BLOCK BYTE: 0x000001c5
[    1.684101] Creating 17 MTD partitions on "7980000.nand":
[    1.689447] 0x000000000000-0x000000140000 : "sbl"
[    1.694867] 0x000000140000-0x000000280000 : "mibib"
[    1.699560] 0x000000280000-0x000000e80000 : "efs2"
[    1.704408] 0x000000e80000-0x000000f40000 : "tz"
[    1.708934] 0x000000f40000-0x000000fa0000 : "rpm"
[    1.713625] 0x000000fa0000-0x000001000000 : "aboot"
[    1.718582] 0x000001000000-0x0000017e0000 : "boot"
[    1.723281] 0x0000017e0000-0x000002820000 : "scrub"
[    1.728174] 0x000002820000-0x000005020000 : "modem"
[    1.732968] 0x000005020000-0x000005420000 : "rfbackup"
[    1.738156] 0x000005420000-0x000005820000 : "oem"
[    1.742770] 0x000005820000-0x000005f00000 : "recovery"
[    1.747972] 0x000005f00000-0x000009100000 : "cache"
[    1.752787] 0x000009100000-0x000009a40000 : "recoveryfs"
[    1.758389] 0x000009a40000-0x00000aa40000 : "cdrom"
[    1.762967] 0x00000aa40000-0x00000ba40000 : "jrdresource"
[    1.768407] 0x00000ba40000-0x000010000000 : "system"
[    1.773239] msm_nand_probe: NANDc phys addr 0x7980000, BAM phys addr 0x7984000, BAM IRQ 164
[    1.781074] msm_nand_probe: Allocated DMA buffer at virt_addr 0xc7840000, phys_addr 0x87840000
[    1.791872] PPP generic driver version 2.4.2
[    1.801126] cnss_sdio 87a00000.qcom,cnss-sdio: CNSS SDIO Driver registered
[    1.807554] msm_otg 78d9000.usb: msm_otg probe
[    1.813333] msm_otg 78d9000.usb: OTG regs = c88f8000
[    1.820702] gbridge_init: gbridge_init successs.
[    1.826344] msm_otg 78d9000.usb: phy_reset: success
[    1.830294] qcom,qpnp-rtc qpnp-rtc-c7307000: rtc core: registered qpnp_rtc as rtc0
[    1.838474] i2c /dev entries driver
[    1.842459] unable to find DT imem DLOAD mode node
[    1.846588] unable to find DT imem EDLOAD mode node
[    1.851332] unable to find DT imem dload-type node
[    1.856921] bq24295-charger 4-006b: bq24295 probe enter
[    1.861161] qcom,iterm-ma = 128
[    1.864476] bq24295_otg_vreg: no parameters
[    1.868502] charger_core_register: Charger Core Version 5.0.0(Built at 20151202-21:36)!
[    1.877007] i2c-msm-v2 78b8000.i2c: msm_bus_scale_register_client(mstr-id:86):0x3 (ok)
[    1.885559] bq24295-charger 4-006b: bq24295_set_bhot_mode 3
[    1.890150] bq24295-charger 4-006b: power_good is 1,vbus_stat is 2
[    1.896588] bq24295-charger 4-006b: bq24295_set_thermal_threshold 100
[    1.902952] bq24295-charger 4-006b: bq24295_set_sys_min 3700
[    1.908639] bq24295-charger 4-006b: bq24295_set_max_target_voltage 4150
[    1.915223] bq24295-charger 4-006b: bq24295_set_recharge_threshold 300
[    1.922119] bq24295-charger 4-006b: bq24295_set_terminal_current_limit iterm_disabled=0, iterm_ma=128
[    1.930917] bq24295-charger 4-006b: bq24295_set_precharge_current_limit bdi->prech_cur=128
[    1.940038] bq24295-charger 4-006b: bq24295_set_safty_timer 0
[    1.945088] bq24295-charger 4-006b: bq24295_set_input_voltage_limit 4520
[    1.972949] sdhci: Secure Digital Host Controller Interface driver
[    1.978151] sdhci: Copyright(c) Pierre Ossman
[    1.982441] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.989092] sdhci_msm 7824900.sdhci: sdhci_msm_probe: ICE device is not enabled
[    1.995473] sdhci_msm 7824900.sdhci: No vreg data found for vdd
[    2.001530] sdhci_msm 7824900.sdhci: sdhci_msm_pm_qos_parse_irq: error -22 reading irq cpu
[    2.009809] sdhci_msm 7824900.sdhci: sdhci_msm_pm_qos_parse: PM QoS voting for IRQ will be disabled
[    2.018600] sdhci_msm 7824900.sdhci: sdhci_msm_pm_qos_parse: PM QoS voting for cpu group will be disabled
[    2.030541] sdhci_msm 7824900.sdhci: sdhci_msm_probe: sdiowakeup_irq = 353
[    2.036867] sdhci_msm 7824900.sdhci: No vmmc regulator found
[    2.042027] sdhci_msm 7824900.sdhci: No vqmmc regulator found
[    2.048266] mmc0: SDHCI controller on 7824900.sdhci [7824900.sdhci] using 32-bit ADMA in legacy mode
[    2.080401] Welcome to pca955x_probe!!
[    2.084362] leds-pca955x 3-0020: leds-pca955x: Using pca9555 16-bit LED driver at slave address 0x20
[    2.095400] sdhci_msm 7824900.sdhci: card claims to support voltages below defined range
[    2.103125] i2c-msm-v2 78b7000.i2c: msm_bus_scale_register_client(mstr-id:86):0x5 (ok)
[    2.114183] msm_otg 78d9000.usb: Avail curr from USB = 1500
[    2.120251] come to USB_SDP_CHARGER!
[    2.123215] Welcome to sn3199_probe!
[    2.126718] leds-sn3199 5-0064: leds-sn3199: Using sn3199 9-bit LED driver at slave address 0x64
[    2.136511] sn3199->led_en_gpio=21
[    2.139143] i2c-msm-v2 78b9000.i2c: msm_bus_scale_register_client(mstr-id:86):0x6 (ok)
[    2.150207] usbcore: registered new interface driver usbhid
[    2.154864] usbhid: USB HID core driver
[    2.159825] sps:BAM 0x078c4000 is registered.
[    2.163573] bimc-bwmon 408000.qcom,cpu-bwmon: BW HWmon governor registered.
[    2.171080] devfreq soc:qcom,cpubw: Couldn't update frequency transition information.
[    2.178513] coresight-fuse a601c.fuse: QPDI fuse not specified
[    2.184242] coresight-fuse a601c.fuse: Fuse initialized
[    2.192407] coresight-csr 6001000.csr: CSR initialized
[    2.197263] coresight-tmc 6026000.tmc: Byte Counter feature enabled
[    2.203204] sps:BAM 0x06084000 is registered.
[    2.207301] coresight-tmc 6026000.tmc: TMC initialized
[    2.212681] coresight-tmc 6025000.tmc: TMC initialized
[    2.220071] nidnt boot config: 0
[    2.224563] mmc0: new ultra high speed SDR50 SDIO card at address 0001
[    2.231120] coresight-tpiu 6020000.tpiu: NIDnT on SDCARD only mode
[    2.236440] coresight-tpiu 6020000.tpiu: TPIU initialized
[    2.242808] coresight-replicator 6024000.replicator: REPLICATOR initialized
[    2.249372] coresight-stm 6002000.stm: STM initialized
[    2.255034] coresight-hwevent 606c000.hwevent: Hardware Event driver initialized
[    2.262312] Netfilter messages via NETLINK v0.30.
[    2.266306] nf_conntrack version 0.5.0 (920 buckets, 3680 max)
[    2.272312] ctnetlink v0.93: registering with nfnetlink.
[    2.277565] ip_set: protocol 6
[    2.280568] ip_tables: (C) 2000-2006 Netfilter Core Team
[    2.285723] arp_tables: (C) 2002 David S. Miller
[    2.290146] TCP: cubic registered
[    2.293915] NET: Registered protocol family 10
[    2.298740] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    2.303407] sit: IPv6 over IPv4 tunneling driver
[    2.308481] NET: Registered protocol family 17
[    2.312340] bridge: automatic filtering via arp/ip/ip6tables has been deprecated. Update your scripts to load br_netfilter if you need this.
[    2.325094] Bridge firewalling registered
[    2.328930] Ebtables v2.0 registered
[    2.333260] NET: Registered protocol family 27
[    2.341362] battery_core_register: Battery Core Version 5.0.0(Built at 20151202-21:36)!
[    2.348466] pmu_battery_probe: vbat_channel=21, tbat_channel=17
[    2.420236] ubi0: attaching mtd16
[    2.723941] ubi0: scanning is finished
[    2.732997] ubi0: attached mtd16 (name "system", size 69 MiB)
[    2.737783] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[    2.744601] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[    2.751333] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
[    2.758540] ubi0: good PEBs: 556, bad PEBs: 2, corrupted PEBs: 0
[    2.764305] ubi0: user volume: 3, internal volumes: 1, max. volumes count: 128
[    2.771476] ubi0: max/mean erase counter: 192/64, WL threshold: 4096, image sequence number: 35657280
[    2.780708] ubi0: available PEBs: 0, total reserved PEBs: 556, PEBs reserved for bad PEB handling: 38
[    2.789921] ubi0: background thread "ubi_bgt0d" started, PID 96
[    2.796395] android_bind cdev: 0xC6583E80, name: ci13xxx_msm
[    2.801508] file system registered
[    2.804974] mbim_init: initialize 1 instances
[    2.809228] mbim_init: Initialized 1 ports
[    2.815074] rndis_qc_init: initialize rndis QC instance
[    2.819713] jrd device_desc.bcdDevice: [0x0242]
[    2.823779] android_bind scheduled usb start work: name: ci13xxx_msm
[    2.830230] android_usb gadget: android_usb ready
[    2.834845] msm_hsusb msm_hsusb: [ci13xxx_start] hw_ep_max = 32
[    2.840741] msm_hsusb msm_hsusb: CI13XXX_CONTROLLER_RESET_EVENT received
[    2.847433] msm_hsusb msm_hsusb: CI13XXX_CONTROLLER_UDC_STARTED_EVENT received
[    2.855851] input: gpio-keys as /devices/soc:gpio_keys/input/input1
[    2.861452] qcom,qpnp-rtc qpnp-rtc-c7307000: setting system clock to 1970-01-01 06:36:41 UTC (23801)
[    2.870315] open file error /usb_conf/usb_config.ini
[    2.876412] jrd_usb_start_work open file erro /usb_conf/usb_config.ini, retry_count:0
[    2.884324] parse_legacy_cluster_params(): Ignoring cluster params
[    2.889468] ------------[ cut here ]------------
[    2.894186] WARNING: CPU: 0 PID: 1 at /home/linux3/jrd/yanping.an/ee40/0810/MDM9607.LE.1.0-00130/apps_proc/oe-core/build/tmp-glibc/work-shared/mdm9607/kernel-source/drivers/cpuidle/lpm-levels-of.c:739 parse_cluster+0xb50/0xcb4()
[    2.914366] Modules linked in:
[    2.917339] CPU: 0 PID: 1 Comm: swapper Not tainted 3.18.20 #1
[    2.923171] [<c00132ac>] (unwind_backtrace) from [<c0011460>] (show_stack+0x10/0x14)
[    2.931092] [<c0011460>] (show_stack) from [<c001c6ac>] (warn_slowpath_common+0x68/0x88)
[    2.939175] [<c001c6ac>] (warn_slowpath_common) from [<c001c75c>] (warn_slowpath_null+0x18/0x20)
[    2.947895] [<c001c75c>] (warn_slowpath_null) from [<c034e180>] (parse_cluster+0xb50/0xcb4)
[    2.956189] [<c034e180>] (parse_cluster) from [<c034b6b4>] (lpm_probe+0xc/0x1d4)
[    2.963527] [<c034b6b4>] (lpm_probe) from [<c024857c>] (platform_drv_probe+0x30/0x7c)
[    2.971380] [<c024857c>] (platform_drv_probe) from [<c0246d54>] (driver_probe_device+0xb8/0x1e8)
[    2.980118] [<c0246d54>] (driver_probe_device) from [<c0246f30>] (__driver_attach+0x68/0x8c)
[    2.988467] [<c0246f30>] (__driver_attach) from [<c02455d0>] (bus_for_each_dev+0x6c/0x90)
[    2.996626] [<c02455d0>] (bus_for_each_dev) from [<c02465a4>] (bus_add_driver+0xe0/0x1c8)
[    3.004786] [<c02465a4>] (bus_add_driver) from [<c02477bc>] (driver_register+0x9c/0xe0)
[    3.012739] [<c02477bc>] (driver_register) from [<c080c3d8>] (lpm_levels_module_init+0x14/0x38)
[    3.021459] [<c080c3d8>] (lpm_levels_module_init) from [<c0008980>] (do_one_initcall+0xf8/0x1a0)
[    3.030217] [<c0008980>] (do_one_initcall) from [<c07e7d4c>] (kernel_init_freeable+0xf0/0x1b0)
[    3.038818] [<c07e7d4c>] (kernel_init_freeable) from [<c0582d48>] (kernel_init+0x8/0xe4)
[    3.046888] [<c0582d48>] (kernel_init) from [<c000dda0>] (ret_from_fork+0x14/0x34)
[    3.054432] ---[ end trace e9ec50b1ec4c8f73 ]---
[    3.059012] ------------[ cut here ]------------
[    3.063604] WARNING: CPU: 0 PID: 1 at /home/linux3/jrd/yanping.an/ee40/0810/MDM9607.LE.1.0-00130/apps_proc/oe-core/build/tmp-glibc/work-shared/mdm9607/kernel-source/drivers/cpuidle/lpm-levels-of.c:739 parse_cluster+0xb50/0xcb4()
[    3.083858] Modules linked in:
[    3.086870] CPU: 0 PID: 1 Comm: swapper Tainted: G        W      3.18.20 #1
[    3.093814] [<c00132ac>] (unwind_backtrace) from [<c0011460>] (show_stack+0x10/0x14)
[    3.101575] [<c0011460>] (show_stack) from [<c001c6ac>] (warn_slowpath_common+0x68/0x88)
[    3.109641] [<c001c6ac>] (warn_slowpath_common) from [<c001c75c>] (warn_slowpath_null+0x18/0x20)
[    3.118412] [<c001c75c>] (warn_slowpath_null) from [<c034e180>] (parse_cluster+0xb50/0xcb4)
[    3.126745] [<c034e180>] (parse_cluster) from [<c034b6b4>] (lpm_probe+0xc/0x1d4)
[    3.134126] [<c034b6b4>] (lpm_probe) from [<c024857c>] (platform_drv_probe+0x30/0x7c)
[    3.141906] [<c024857c>] (platform_drv_probe) from [<c0246d54>] (driver_probe_device+0xb8/0x1e8)
[    3.150702] [<c0246d54>] (driver_probe_device) from [<c0246f30>] (__driver_attach+0x68/0x8c)
[    3.159120] [<c0246f30>] (__driver_attach) from [<c02455d0>] (bus_for_each_dev+0x6c/0x90)
[    3.167285] [<c02455d0>] (bus_for_each_dev) from [<c02465a4>] (bus_add_driver+0xe0/0x1c8)
[    3.175444] [<c02465a4>] (bus_add_driver) from [<c02477bc>] (driver_register+0x9c/0xe0)
[    3.183398] [<c02477bc>] (driver_register) from [<c080c3d8>] (lpm_levels_module_init+0x14/0x38)
[    3.192107] [<c080c3d8>] (lpm_levels_module_init) from [<c0008980>] (do_one_initcall+0xf8/0x1a0)
[    3.200877] [<c0008980>] (do_one_initcall) from [<c07e7d4c>] (kernel_init_freeable+0xf0/0x1b0)
[    3.209475] [<c07e7d4c>] (kernel_init_freeable) from [<c0582d48>] (kernel_init+0x8/0xe4)
[    3.217542] [<c0582d48>] (kernel_init) from [<c000dda0>] (ret_from_fork+0x14/0x34)
[    3.225090] ---[ end trace e9ec50b1ec4c8f74 ]---
[    3.229667] /soc/qcom,lpm-levels/qcom,pm-cluster@0: No CPU phandle, assuming single cluster
[    3.239954] qcom,cc-debug-mdm9607 1800000.qcom,debug: Registered Debug Mux successfully
[    3.247619] emac_lan_vreg: disabling
[    3.250507] mem_acc_corner: disabling
[    3.254196] clock_late_init: Removing enables held for handed-off clocks
[    3.262690] ALSA device list:
[    3.264732]   No soundcard�[    3.274083] UBIFS (ubi0:0): background thread "ubifs_bgt0_0" started, PID 102
[    3.305224] UBIFS (ubi0:0): recovery needed
[    3.466156] UBIFS (ubi0:0): recovery completed
[    3.469627] UBIFS (ubi0:0): UBIFS: mounted UBI device 0, volume 0, name "rootfs"
[    3.476987] UBIFS (ubi0:0): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
[    3.486876] UBIFS (ubi0:0): FS size: 45838336 bytes (43 MiB, 361 LEBs), journal size 9023488 bytes (8 MiB, 72 LEBs)
[    3.497417] UBIFS (ubi0:0): reserved for root: 0 bytes (0 KiB)
[    3.503078] UBIFS (ubi0:0): media format: w4/r0 (latest is w4/r0), UUID 4DBB2F12-34EB-43B6-839B-3BA930765BAE, small LPT model
[    3.515582] VFS: Mounted root (ubifs filesystem) on device 0:12.
[    3.520940] Freeing unused kernel memory: 276K (c07e7000 - c082c000)
INIT: version 2.88 booting

18 February, 2021 06:44PM

Russ Allbery

Review: Solutions and Other Problems

Review: Solutions and Other Problems, by Allie Brosh

Publisher: Gallery Books
Copyright: September 2020
ISBN: 1-9821-5694-5
Format: Hardcover
Pages: 519

Solutions and Other Problems is the long-awaited second volume of Allie Brosh's work, after the amazing Hyperbole and a Half. The first collection was a mix of original material and pieces that first appeared on her blog. This is all new work, although one of the chapters is now on her blog as a teaser.

As with all of Brosh's previous work, Solutions and Other Problems is mostly drawings (in her highly original, deceptively simple style) with a bit of prose in between. It's a similar mix of childhood stories, off-beat interpretations of day-to-day life, and deeper and more personal topics. But this is not the same type of book as Hyperbole and a Half, in a way that is hard to capture in a review.

When this book was postponed and then temporarily withdrawn, I suspected that something had happened to Brosh. I was hoping that it was just the chaos of her first book publication, but, sadly, no. We find out about some of what happened in Solutions and Other Problems, in varying amounts of detail, and it's heart-wrenching. That by itself gives the book a more somber tone.

But, beyond that, I think Solutions and Other Problems represents a shift in mood and intention. The closest I can come to it is to say that Hyperbole and a Half felt like Brosh using her own experiences as a way to tell funny stories, and this book feels like Brosh using funny stories to talk about her experiences. There are still childhood hijinks and animal stories mixed in, but even those felt more earnest, more sad, and less assured or conclusive. This is in no way a flaw, to be clear; just be aware that if you were expecting more work exactly like Hyperbole and a Half, this volume is more challenging and a bit more unsettling.

This does not mean Brosh's trademark humor is gone. Chapter seventeen, "Loving-Kindness Exercise," is one of the funniest things I've ever read. "Neighbor Kid" captures my typical experience of interacting with children remarkably well. And there are, of course, more stories about not-very-bright pets, including a memorable chapter ("The Kangaroo Pig Gets Drunk") on just how baffling our lives must be to the animals around us. But this book is more serious, even when there's humor and absurdity layered on top, and anxiety felt like a constant companion.

As with her previous book, many of the chapters are stories from Brosh's childhood. I have to admit this is not my favorite part of Brosh's work, and the stories in this book in particular felt a bit less funny and somewhat more uncomfortable and unsettling. This may be a very individual reaction; you can judge your own in advance by reading "Richard," the second chapter of the book, which Brosh posted to her blog. I think it's roughly typical of the childhood stories here.

The capstone of Hyperbole and a Half was Brosh's fantastic two-part piece on depression, which succeeded in being hilarious and deeply insightful at the same time. I think the capstone of Solutions and Other Problems is the last chapter, "Friend," which is about being friends with yourself. For me, it was a good encapsulation of both the merits of this book and the difference in tone. It's less able to find obvious humor in a psychological struggle, but it's just as empathetic and insightful. The ending is more ambiguous and more conditional; the tone is more wistful. It felt more personal and more raw, and therefore a bit less generalized. Her piece on depression made me want to share it with everyone I knew; this piece made me want to give Brosh a virtual hug and tell her I'm glad she's alive and exists in the world. That about sums up my reaction to this book.

I bought Solutions and Other Problems in hardcover because I think this sort of graphic work benefits from high-quality printing, and I was very happy with that decision. Gallery Books used heavy, glossy paper and very clear printing. More of the text is outside of the graphic panels than I remember from the previous book. I appreciated that; I thought it made the stories much easier to read. My one quibble is that Brosh does use fairly small lettering in some of the panels and the color choices and the scrawl she uses for stylistic reasons sometimes made that text difficult for me to read. In those few places, I would have appreciated the magnifying capabilities of reading on a tablet.

I don't think this is as good as Hyperbole and a Half, but it is still very good and very worth reading. It's harder reading, though, and you'll need to brace yourself more than you did before. If you're new to Brosh, start with Hyperbole and a Half, or with the blog, but if you liked those, read this too.

Rating: 8 out of 10

18 February, 2021 05:19AM

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

dang 0.0.13: New intradayMarketMonitor

sp500 intraday monitor

A new release of the dang package got to CRAN earlier today, a few months since the last relase. The dang package regroups a few functions of mine that had no other home as for example lsos() from a StackOverflow question from 2009 (!!) is one, this overbought/oversold price band plotter from an older blog post is another.

This release adds one function I tweeted about one month ago. It takes a function Josh Ulrich originally tweeted about in November with a reference to this gist. I refactored this into a proper functions and polished a few edges: the data now properly rolls off after a fixed delay (of two days), should work with other symbols (though we both focused on ^GSPC as a free (!!) real-time SP500 index (albeit only during trading hours), properly gaps between trading days and more. You can simply invoke it via


and a chart just like the one here will grow (though there is no “state”: if you stop it, or reboot, or … the plot starts from scratch).

The short NEWS entry follows.

Changes in version 0.0.13 (2021-02-17)

  • New function intradayMarketMonitor based on an earlier gist-posted snippet by Josh Ulrich.

  • The CI setup was generalized as a test for 'r-ci' and is used essentially unchanged with three different providers.

Courtesy of my CRANberries, there is a comparison to the previous release. For questions or comments use the issue tracker off the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

18 February, 2021 01:42AM

February 17, 2021

hackergotchi for Norbert Preining

Norbert Preining

Debian KDE/Plasma Status 2021-02-18

Lots of time has passed since the last status update, and Debian is going into pre-release freeze, so let us report a bit about the most recent changes: Debian/bullseye will have Plasma 5.20.5, Frameworks 5.78, Apps 20.12. Debian/experimental already carries Plasma 5.21 and Frameworks 5.79, and that is also the level at the OSC builds.

Debian Bullseye

We are in soft freeze now, and only targeted fixes are allowed, but Bullseye is carrying a good mixture consisting of the KDE Frameworks 5.78, including several backports of fixes from 5.79 to get smooth operation. Plasma 5.20.5, again with several cherry picks for bugs will be in Bullseye, too. The KDE/Apps are mostly at 20.12 level, and the KDE PIM group packages (akonadi, kmail, etc) are at 20.08.

Debian experimental

In the last days I have uploaded frameworks 5.79 and Plasma 5.21 to Debian/experimental. For Plasma there is still some NEW processing to be done, but in due time the packages will be available and installable from experimental.

OBS packages

The OBS packages as usual follow the latest release, and currently ship KDE Frameworks 5.79, KDE Apps 20.12.2, and Plasma 5.21.0. The package sources are as usual (note the different path for the Plasma packages and the App packages, containing the release version!), for Debian/unstable:

deb https://download.opensuse.org/repositories/home:/npreining:/debian-kde:/frameworks/Debian_Unstable/ ./
deb https://download.opensuse.org/repositories/home:/npreining:/debian-kde:/plasma521/Debian_Unstable/ ./
deb https://download.opensuse.org/repositories/home:/npreining:/debian-kde:/apps2012/Debian_Unstable/ ./
deb https://download.opensuse.org/repositories/home:/npreining:/debian-kde:/other/Debian_Unstable/ ./

and the same with Testing instead of Unstable for Debian/testing.

Digikam beta

There is also a separate repository for the upcoming digikam release:

deb https://download.opensuse.org/repositories/home:/npreining:/debian-kde:/digikam-beta/Debian_Unstable/ ./

just in case you want to test the rc release of digikam 7.2.0.

17 February, 2021 09:08PM by Norbert Preining

hackergotchi for Martin-&#201;ric Racine

Martin-Éric Racine

OpenWRT: WRT54GL: Backfire: IPv6 issues

While having a Debian boxen as a router feels nice, I kept on longing for something smaller and quieter. I then remembered that I still had my old WRT54GL somewhere. After upgrading the OpenWRT firmware to the latest supported version for that hardware (Backfire 10.03.1, r29592), I installed radvd and wide-dhcpv6-client. Configuring radvd to deliver consistent results was easy enough.

The issue I keep on experiencing is the external interface (wan) dropping the IPv6 address it received from the ISP via router advertisement, which in turn kills the default IPv6 route to the outside world. Logging in via SSH and manually running "rdisc6 eth0.1" restores the IPv6 gateway. I just honestly wished I didn't have to do this every time I need to reboot the router.

Does this issue sound familiar to anyone? What was the solution?

PS: No, I won't just go and ditch this WRT54GL just because new toys exist on the market. This is obviously a software issue, so I need a software solution.

PPS: IPv6 pretty much works out of the box on the Debian boxen I had been using as my router. I previously wrote about this on my blog. Basically, it's unlikely to be an ISP issue.

17 February, 2021 06:09PM by Martin-Éric (noreply@blogger.com)

hackergotchi for Louis-Philippe Véronneau

Louis-Philippe Véronneau

What are the incentive structures of Free Software?

When I started my Master's degree in January 2018, I was confident I would be done in a year and half. After all, I only had one year of classes and I figured 6 months to write a thesis would be plenty.

Three years later, I'm finally done: the final version of my thesis was accepted on January 22nd 2021.

My thesis, entitled What are the incentive structures of Free Software? An economic analysis of Free Software's specific development model, can be found here 1. If you care about such things, both the data and the final document can be built from source with the code in this git repository.

Results and analysis

My thesis is divided in four main sections:

  1. an introduction to FOSS
  2. a chapter discussing the incentive structures of Free Software (and arguing the so called “Tragedy of the Commons” isn't inevitable)
  3. a chapter trying to use empirical data to validate the theories presented in the previous chapter
  4. an annex on the various FOSS business models

If you're reading this blog post, chances are you'll find both section 1 and 4 a tad boring, as you might already be familiar with these concepts.


So, why do people contribute to Free Software? Unsurprisingly, it's complicated. Many economists have studied this topic, but for some reason, most research happened in the early 2000s.

Although papers don't all agree with each other and most importantly, about the variables' importance, the main incentives2 can be summarized by:

  • expectation of monetary gain
  • writing FOSS as a hobby (that includes “scratching your own itch”)
  • liking the FOSS community and feeling a sense of belonging
  • altruism (writing FOSS for Good™)

Giving weights to these variables is not an easy thing: the FOSS ecosystem is highly heterogeneous and thus, people tend to write FOSS for different reasons. Moreover, incentives tend to shift with time as the ecosystem does. People writing Free Software in the 1990s probably did it for different reasons than people in 2021.

These four variables can also be divided in two general categories: extrinsic and intrinsic incentives. Monetary gain expectancy is an extrinsic incentive (its value is delayed and mediated), whereas the three other ones are intrinsic (they have an immediate value by themselves).

Empirical analysis

Theory is nice, but it's even better when you can back it up with data. Sadly, most of the papers on the economic incentives of FOSS are either purely theoretical, or use sample sizes so small they could as well be.

Using the data from the StackOverflow 2018 survey, I thus tried to see if I could somehow confirm my previous assumptions.

With 129 questions and more than 100 000 respondents (which after statistical processing yields between 28 000 and 39 000 observations per variable of interest), the StackOverflow 2018 survey is a very large dataset compared to what economists are used to work with.

Sadly, it wasn't entirely enough to come up with hard answers. There is a strong and significant correlation between writing Free Software and having a higher salary, but endogeneity problems3 made it hard to give a reliable estimate of how much money this would represent. Same goes for writing code has a hobby: it seems there is a strong and significant correlation, but the exact numbers I came up with cannot really be trusted.

The results on community as an incentive to writing FOSS were the ones that surprised me the most. Although I expected the relation to be quite strong, the coefficients predicted were in fact quite small. I theorise this is partly due to only 8% of the respondents declaring they didn't feel like they belonged in the IT community. With such a high level of adherence, the margin for improvement has to be smaller.

As for altruism, I wasn't able get any meaningful results. In my opinion this is mostly due to the fact there was no explicit survey question on this topic and I tried to make up for it by cobbling data together.

Kinda anti-climatic, isn't it? I would've loved to come up with decisive conclusions on this topic, but if there's one thing I learned while writing this thesis, it is I don't know much after all.

  1. Note that the thesis is written in French. 

  2. Of course, life is complex and so are people's motivations. One could come up with dozen more reasons why people contribute to Free Software. The "fun" of theoretical modelisation is trying to make complex things somewhat simpler. 

  3. I'll spare you the details, but this means there is no way to know if this correlation is the result of a causal link between the two variables. There are ways to deal with this problem (using an instrumental variables model is a very popular one), but again, the survey didn't provide the proper instruments to do so. For example, it could very well be the correlation is due to omitted variables. If you are interested in this topic (and can read French), I talk about this issue in section 3.2.8. 

17 February, 2021 05:00AM by Louis-Philippe Véronneau

February 16, 2021

Vincent Fourmond

QSoas tips and tricks: permanently storing meta-data

It is one thing to acquire and process data, but the data themselves are most often useless without the context, the conditions in which the experiments were made. These additional informations can be called meta-data. In a previous post, we have already described how one can set meta-data to data that are already loaded, and how one can make use of them.

QSoas is already able to figure out some meta-data in the case of electrochemical data, most notably in the case of files acquired by GPES, ECLab or CHI potentiostats. However, only a small number of constructors are supported as of now[1], and there are a number of experimental details that the software is never going to be able to figure out for you, such as the pH, the sample, what you were doing...

The new version of QSoas provides a means to permanently store meta-data for experimental data files:
QSoas> record-meta pH 7 file.dat
This command uses record-meta to permanently store the information pH = 7 for the file file.dat. Any time QSoas loads the file again, either today or in one year, the meta-data will contain the value 7 for the field pH. Behind the scenes, QSoas creates a single small file, file.dat.qsm, in which the meta-data are stored (in the form of a JSON dictionnary).

You can set the same meta-data to many files in one go, using wildcards (see load for more information). For instance, to set the pH=7 meta-data to all the .dat files in the current directory, you can use:
QSoas> record-meta pH 7 *.dat
You can only set one meta-data for each call to record-meta, but you can use it as many times as you like.

Finally, you can use the /for-which option to load or browse to select only the files which have the meta you need:
QSoas> browse /for-which=$meta.pH<=7
This command browses the files in the current directory, showing only the ones that have a pH meta-data which is 7 or below.

[1] I'm always ready to implement the parsing of other file formats that could be useful for you. If you need parsing of special files, please contact me, sending the given files and the meta-data you'd expect to find in those.

About QSoas

QSoas is a powerful open source data analysis program that focuses on flexibility and powerful fitting capacities. It is released under the GNU General Public License. It is described in Fourmond, Anal. Chem., 2016, 88 (10), pp 5050–5052. Current version is 3.0. You can download its source code there (or clone from the GitHub repository) and compile it yourself, or buy precompiled versions for MacOS and Windows there.

16 February, 2021 10:51AM by Vincent Fourmond (noreply@blogger.com)

hackergotchi for Michael Prokop

Michael Prokop

How to properly use 3rd party Debian repository signing keys with apt

(Blogging this, since this is a recurring anti-pattern I noticed at several customers and often comes up during deployments of 3rd party repositories.)

Update on 2021-02-19: clarified, that Signed-By requires apt >= 1.1, thanks Vincent Bernat

Many upstream projects provide Debian repository instructions like this:

curl -fsSL https://example.com/stable/debian.gpg | sudo apt-key add -

Do not follow this, for different reasons, including:

  1. You do not see what you get before adding the GPG key to your global apt trust store
  2. You can’t easily script this via your preferred configuration management (the apt-key manpage clearly discourages programmatic usage)
  3. The signing key is considered valid for all your enabled Debian repositories (instead of only a specific one)
  4. You need GnuPG (either gnupg2 or gnupg1) on your system for usage with apt-key

There’s a much better approach to this: download the GPG key, make sure it’s in the appropriate format, then use it via `deb [signed-by=/usr/share/keyrings/…]` in your apt’s sources list configuration. Note and FTR: the Signed-By feature is available starting with apt 1.1 (so apt in Debian jessie/8 and older does not support it).


  • Install GPG keys in ascii-armored / old public key block format as /usr/share/keyrings/example.asc and use `deb [signed-by=/usr/share/keyrings/example.asc] https://example.com/…` in apt’s sources.list configuration
  • Install GPG keys in binary OpenPGP format as /usr/share/keyrings/example.gpg and use `deb [signed-by=/usr/share/keyrings/example.gpg] https://example.com/…` in apt’s sources.list configuration

As an example, let’s demonstrate this with the Tailscale Debian repository for buster.
Downloading the GPG file will give you an ascii-armored GPG file:

% curl -fsSL -o buster.gpg https://pkgs.tailscale.com/stable/debian/buster.gpg
% gpg --keyid-format long buster.gpg 
gpg: WARNING: no command supplied.  Trying to guess what you mean ...
pub   rsa4096/458CA832957F5868 2020-02-25 [SC]
uid                           Tailscale Inc. (Package repository signing key) <info@tailscale.com>
sub   rsa4096/B1547A3DDAAF03C6 2020-02-25 [E]
% file buster.gpg
buster.gpg: PGP public key block Public-Key (old)

If you have apt version >= 1.4 available (Debian >=stretch/9 and Ubuntu >=bionic/18.04), you can use this file directly as follows:

% sudo mv buster.gpg /usr/share/keyrings/tailscale.asc
% cat /etc/apt/sources.list.d/tailscale.list
deb [signed-by=/usr/share/keyrings/tailscale.asc] https://pkgs.tailscale.com/stable/debian buster main
% sudo apt update

And you’re done!

Iff your apt version really is older than 1.4, you need to convert the ascii-armored GPG file into a GPG key public ring file (AKA binary OpenPGP format), either by just dearmor-ing it (if you don’t care about checking ID + fingerprint):

% gpg --dearmor < buster.gpg > tailscale.gpg

or if you prefer to go via GPG, you can also use a temporary GPG home directory (if you don’t care about going through your personal GPG setup):

% mkdir --mode=700 /tmp/gpg-tmpdir
% gpg --homedir /tmp/gpg-tmpdir --import ./buster.gpg
gpg: keybox '/tmp/gpg-tmpdir/pubring.kbx' created
gpg: /tmp/gpg-tmpdir/trustdb.gpg: trustdb created
gpg: key 458CA832957F5868: public key "Tailscale Inc. (Package repository signing key) <info@tailscale.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1
% gpg --homedir /tmp/gpg-tmpdir --output tailscale.gpg  --export-options=export-minimal --export 0x458CA832957F5868
% rm -rf /tmp/gpg-tmpdir

The resulting GPG key public ring file should look like that:

% file tailscale.gpg 
tailscale.gpg: PGP/GPG key public ring (v4) created Tue Feb 25 04:51:20 2020 RSA (Encrypt or Sign) 4096 bits MPI=0xc00399b10bc12858...
% gpg tailscale.gpg 
gpg: WARNING: no command supplied.  Trying to guess what you mean ...
pub   rsa4096/458CA832957F5868 2020-02-25 [SC]
uid                           Tailscale Inc. (Package repository signing key) <info@tailscale.com>
sub   rsa4096/B1547A3DDAAF03C6 2020-02-25 [E]

Then you can use this GPG file on your system as follows:

% sudo mv tailscale.gpg /usr/share/keyrings/tailscale.gpg
% cat /etc/apt/sources.list.d/tailscale.list
deb [signed-by=/usr/share/keyrings/tailscale.gpg] https://pkgs.tailscale.com/stable/debian buster main
% sudo apt update

Such a setup ensures:

  1. You can verify the GPG key file (ID + fingerprint)
  2. You can easily ship files via /usr/share/keyrings/ and refer to it in your deployment scripts, configuration management,… (and can also easily update or get rid of them again!)
  3. The GPG key is valid only for the repositories with the corresponding `[signed-by=/usr/share/keyrings/…]` entry
  4. You don’t need to install GnuPG (neither gnupg2 nor gnupg1) on the system which is using the 3rd party Debian repository

Thanks: Guillem Jover for reviewing an early draft of this blog article.

16 February, 2021 09:19AM by mika

February 15, 2021

Free Software Fellowship

Matthias Kirschner, Jonas Oberg & FSFE: Paternity, Maternity, Sick leave hypocrisy

FSFE, Matthias Kirschner, student union, sexism

Today the Fellowship is proud to leak to more messages from the FSFE proving the hypocrisy of Matthias Kirschner.

To recap the sequence of events:

  1. A fellow left a EUR 150,000 bequest to the FSFE.
  2. There was fighting about the money and Matthias Kirschner, President, fell out with Jonas Oberg, Executive Director
  3. To avoid being sacked, Oberg took paternity leave while looking for another job.
  4. Kirschner had a huge fit. He wanted to fire Oberg but he couldn't. So Kirschner took paternity leave too, his message announcing paternity leave is leaked below
  5. When Kirschner came back, he was still pissed off so he decided to remove the elections from the constitution.
  6. The representative elected by the community resigned in disgust
  7. Kirschner was angry that all the serious people quit and he had nobody to blame. So he spread rumours that the community representative, who had already resigned, is a spammer and pedophile
  8. When the two female employees, Susanne Eiswirt and Galia Mancheva, spoke about wage equality, Kirschner sacked them. Finally, he got to fire somebody. Women. In particular, he sacked Galia while she was on sick leave, he didn't give her the same support that people gave Kirschner during his paternity leave episode.

To summarize: the only people Kirschner actually sacked or expelled were women. All the serious people resigned in disgust before Kirschner could get out his knife.

Leak 1: Matthias Kirschner paternity leave

Subject: [GA] New fork / MK off until 23 April
Date: Sun, 25 Mar 2018 07:33:09 +0200
From: Matthias Kirschner <mk@fsfe.org>
To: FSFE General Assembly <ga@lists.fsfe.org>, FSFE EU core team <team@lists.fsfe.org>

This week my child Paul Benjamin Kirschner was born. We are all fine and enjoy those wonderful and unique moments of our lives. Until 23 April I will be offline to spent time with my family.

I am very happy that the FSFE is in a state which enables me to take this time off, knowing that all day to day operations will be performed reliably by our volunteers and staffers. During that time some topics will be on hold, and I will not follow-up any e-mail discussions. Heiki and Patrick will decide if something is important enough to contact me. In case you have issues to discuss which are not absolutely urgent, I ask you to wait with bringing them up, until I am back again.

Thank you for enabling me to enjoy the birth of my child and fully concentrate on my family for the next weeks. All the best from a tired but exuberantly happy,

Matthias Kirschner - President - Free Software Foundation Europe
Schönhauser Allee 6/7, 10119 Berlin, Germany | t +49-30-27595290
Registered at Amtsgericht Hamburg, VR 17030 | (fsfe.org/join)
Contact (fsfe.org/about/kirschner) - Weblog (k7r.eu/blog.html)

Galia Mancheva's comments about her medical leave

Does Kirschner's wife, Kristina, know that Kirschner intruded on Galia's home?

An extract from Galia's full report:

I suspected Matthias was preparing to fire me, and indeed I was on a clock. He needed to make sure he was re-elected FSFE president before he could get rid of me. It would have damaged his chances to have fired all the full-time women in the office in the two months leading up to the elections – an unnecessary risk – but once approved he would have another two years' free reign. At this time my friends were starting to worry – the psychological pressure and lack of enough time off caused my condition was decline. I had to take a sick leave. Certain that I was about to get fired, my friends encouraged me to seek legal counsel, if so only to keep myself focussed on constructive tasks. Immediately after my sick leave announcement, Matthias fired me over the phone on a Friday night, and threatened me to immediately go to the office to hand over work-related items. Appearing in the office during a sick leave is illegal in Germany, so I refused. A weekend of non-stop calls followed, including from hidden phone numbers. He even texted telling me I should answer my phone, for my own better. Even after my lawyer warned him to terminate all attempts to communicate with me and send someone else to pick up my work laptop, he came in person to my house, and was very irritated that I was not alone.

Leak 2: Jonas Oberg paternity leave

Subject: Continuation in volunteer roles
Date: Thu, 8 Feb 2018 09:55:47 +0100
From: Jonas Oberg <jonas@fsfe.org>
To: team@lists.fsfe.org

Hi everyone,

I just reviewed the mailing lists where I was subscribed for the FSFE, and wanted to give you a heads up about what you can expect from me in the coming months (and years!).

From the 14th of February, I will be largely disconnected from the day-to-day operations of the FSFE, owing to my parental leave. I'm still a member of the association, and I still have an interest in many of our activities, so I will continue my work in some areas as a volunteer.

Specifically, I will, as a member, remain subscribed and part of the ga@ and team@ lists, but I will read team@ less seldom and focus my attention to the much needed work in the GA.

I will also remain as a volunteer for the following activities:

- Our Legal team
- The FSFE Nordic local group

In addition, I plan to volunteer time to oversee some of the work with the AMS implementation and system administration. It's not conceivable we will manage a proper handover of all tasks in the time remaining, and I have a personal interest in making sure this is properly done.

This all being said, you can of course always rely on my help or guidance if you need it for any other activity too. Just make sure that you ask me specifically (direct mail) about it, and understand that from the 14th, my ability to respond will be much lower during the days and I'll catch up more infrequently, often in the evenings.

In the coming months, I will also have two weeks when I'll work full time for the FSFE; for the REUSE activities we will have at the Open Source Leadership Summit, and for the LLW in Barcelona. So you can expect more mails from me those weeks (w10 and 16) :-)


Jonas Öberg
Executive Director

FSFE e.V. - keeping the power of technology in your hands. Your
support enables our work, please join us today http://fsfe.org/join

Susanne Eiswirth, Matthias Kirschner, Galia Mancheva, FSFE, workplace bullying, harassment

15 February, 2021 11:55AM

hackergotchi for Axel Beckert

Axel Beckert

Starting a GNU Screen session via SSH's ~/.ssh/config

This is more or less a followup to this blog posting of mine about AutoSSH and GNU Screen from nearly ten years ago — which by the way is still valid and still the way, I use SSH and GNU Screen.

Recently a friend asked me how to automatically start or reconnect to a GNU Screen session directly via OpenSSH’s configuration file. Here’s how to do it:

Add an entry to ~/.ssh/config similar to this one:

Host screen_on_server
    Hostname server.example.org
    RequestTTY yes
    RemoteCommand screen -RD

and then just call ssh screen_on_server and you’ll get connected to an existing screen session if present, otherwise you’ll a new new one.

Should work with tmux, too, but might need commandlien different options.

15 February, 2021 04:50AM by Axel Beckert (abe+blog@deuxchevaux.org)

Russ Allbery

Review: Spheres of Influence

Review: Spheres of Influence, by Ryk E. Spoor

Series: Arenaverse #2
Publisher: Baen
Copyright: November 2013
ISBN: 1-4516-3937-6
Format: Kindle
Pages: 576

Spheres of Influence is a direct sequel to Grand Central Arena, which introduces the world background and is also a much better book. There is a detailed recap of the previous volume (thank you!) and a summary of things that happened between the volumes (that was odd), so it's easy to refresh your memory, but there's no point in reading this book if you've not read the first one.

In this series, Spoor is explicitly writing a throw-back space adventure inspired by E.E. "Doc" Smith and similar SF from the 1920s to the 1950s. Grand Central Arena was the discovery and exploration story, which in my opinion is where that literary tradition is at its strongest. Spheres of Influence veers into a different and less appealing part of that tradition: the moment when the intrepid space explorer is challenged by the ignorant Powers That Be at home, who don't understand the importance of anything that's happening.

Captain Ariane Austin and her crew made a stunning debut into the Arena, successfully navigated its politics (mostly via sheer audacity and luck), and achieved a tentatively strong position for humanity. However, humanity had never intended them to play that role. There isn't much government in Spoor's (almost entirely unexplained) anarcho-libertarian future, but there is enough for political maneuvering and the appointment of a more official ambassador to the Arena who isn't Ariane. But the Arena has its own rules that care nothing about human politics, which gives Ariane substantial leverage to try to prevent Earth politicians from making a mess of things.

This plot could be worse. Unlike his source material, Spoor is not entirely invested in authoritarian politics, and the plot resolution is a bit friendlier to government oversight than one might expect. (It's disturbing, though, that this oversight seems to consist mostly of the military, and it's not clear how those people are selected.) But the tradition of investing vast powers in single people of great moral character is one of the less defensible tropes of early American SF, and Spoor chooses to embrace it to an unfortunate degree. Clearing out all the bureaucratic second-guessing to let the honorable person who has stumbled across vast power make all the decisions is a type of simplistic politics with a long, bad history in US fiction. The author can make it look like a good idea by yanking hard on the scales; Ariane makes all the right decisions because she's the heroine and therefore of course she does. I was unsettled, in this year of 2021, by the book's apparent message that her major failing is her unwillingness to consolidate her power.

This isn't the only problem I had with this book. Before we get to the political maneuvering, the plot takes a substantial digression into the Hyperion Project.

The Hyperion Project showed up in the first book as part of the backstory of one of the characters. I'll omit the details to avoid spoilers, but in the story it functioned as an excuse to model a character directly on E.E. "Doc" Smith characters. The details never seemed that interesting, but as background it was easy to read past, and the character in question was still moderately enjoyable.

Unfortunately, the author was more invested in this bit of background than I was. Spheres of Influence introduces four more characters from the same project, including Wu Kong, a cliched mash-up of numerous different Monkey King stories who becomes a major viewpoint character. (The decision to focus on a westernized, exoticized version of a Chinese character didn't seem that wise to me.) One problem is that Spoor clearly thinks Wu Kong is a more interesting character than I do, but my biggest complaint is that introducing these new characters was both unnecessary and pulled the story away from the pieces I was interested in. I want to read more about the Arena and its politics, alien technology, and awesome vistas, not about some significantly less interesting historical human project devoted to bringing fictional characters to life.

And that's the third problem with this book: not enough happens. Grand Central Arena had a good half-dozen significant plot developments set among some great sense-of-wonder exploration and alien first contact. There are only two major plot events in Spheres of Influence, both are dragged out with unnecessary description and posturing, and neither show us much that's exciting or new. The exploration of the Arena grinds nearly to a halt, postponing the one offered bit of deep exploration for the third book. There are some satisfying twists and turns in the bits of plot we do get, but nothing that justifies nearly 600 pages.

This is not a very good book, and huge step down from the first book of the series. In its defense, it still offers the sort of optimistic (and, to be honest, simplistic) adventure that I was looking for after reading a book full of depressing politics. It's hard not to enjoy the protagonists taking audacious risks, revealing hidden talents, and winning surprising victories. But I wanted the version with more exploration, more new sights, less authoritarian and militaristic politics, and less insertion of fictional characters.

Also, yes, we know that one of the characters is an E.E. "Doc" Smith character. Please give the cliched Smith dialogue tics a rest. All of the "check to nine decimal places" phrases are hard enough to handle in Smith's short and fast-moving books. They're agonizing in a slow-moving book three times as long.

Not recommended, although I'm still invested enough in the setting that I'll probably read the third book when I'm feeling in the mood for some feel-good adventure. It appears to have the plot developments I was hoping would be in this one.

Followed by Challenges of the Deeps.

Rating: 5 out of 10

15 February, 2021 04:35AM

February 14, 2021

Enrico Zini

hackergotchi for Chris Lamb

Chris Lamb

The Silence of the Lambs: 30 Years On

No doubt it was someone's idea of a joke to release Silence of the Lambs on Valentine's Day, thirty years ago today. Although it references Valentines at one point and hints at a deeper relationship between Starling and Lecter, it was clearly too tempting to jeopardise so many date nights. After all, how many couples were going to enjoy their ribeyes medium-rare after watching this?

Given the muted success of Manhunter (1986), Silence of the Lambs was our first real introduction to Dr. Lecter. Indeed, many of the best scenes in this film are introductions: Starling's first encounter with Lecter is probably the best introduction in the whole of cinema, but our preceding introduction to the asylum's factotum carries a lot of cultural weight too, if only because the camera's measured pan around the environment before alighting on Barney has been emulated by so many first-person video games since.

We first see Buffalo Bill at the thirty-two minute mark. (Or, more tellingly, he sees us.) Delaying the viewer's introduction to the film's villain is the mark of a secure and confident screenplay, even if it was popularised by the budget-restricted Jaws (1975) which hides the eponymous shark for one hour and 21 minutes.

It is no mistake that the first thing we see of Starling do is, quite literally, pull herself up out of the unknown. With all of the focus on the Starling—Lecter repartee, the viewer's first introduction to Starling is as underappreciated as she herself is to the FBI. Indeed, even before Starling tells Lecter her innermost dreams, we learn almost everything we need to about Starling in the first few minutes: we see her training on an obstacle course in the forest, the unused rope telling us that she is here entirely voluntarily. And we can surely guess why; the passing grade for a woman in the FBI is to top of the class, and Starling's not going to let an early February in Virginia get in the way of that.

We need to wait a full three minutes before we get our first line of dialogue, and in just eight words ("Crawford wants to see you in his office...") we get our confirmation about the FBI too. With no other information other than he can send a messenger out into the cold, we can intuit that Crawford tends to get what Crawford wants. It's just plain "Crawford" too; everyone knows his actual title, his power, "his" office.

The opening minutes also introduce us to the film's use of visual hierarchy. Our Hermes towers above Starling throughout the brief exchange (she must push herself even to stay within the camera's frame). Later, Starling always descends to meet her demons: to the asylum's basement to visit Lecter and down the stairs to meet Buffalo Bill. Conversely, she feels safe enough to reveal her innermost self to Lecter on the fifth floor of the courthouse. (Bong Joon-ho's Parasite (2019) uses elevation in an analogous way, although a little more subtly.)

The messenger turns to watch Starling run off to Crawford. Are his eyes involuntarily following the movement or he is impressed by Starling's gumption? Or, almost two decades after John Berger's male gaze, is he simply checking her out? The film, thankfully, leaves it to us.

Crawford is our next real introduction, and our glimpse into the film's sympathetic treatment of law enforcement. Note that the first thing that the head of the FBI's Behavioral Science Unit does is to lie to Starling about the reason to interview Lecter, despite it being coded as justified within the film's logic. We learn in the book that even Barney deceives Starling, recording her conversations with Lecter and selling her out to the press. (Buffalo Bill always lies to Starling, of course, but I think we can forgive him for that.) Crawford's quasi-compliment of "You grilled me pretty hard on the Bureau's civil rights record in the Hoover years..." then encourages the viewer to conclude that the FBI's has been a paragon of virtue since 1972... All this (as well as her stellar academic record, Crawford's wielding of Starling's fragile femininity at the funeral home and the cool reception she receives from a power-suited Senator Ruth Martin), Starling must be constantly asking herself what it must take for anyone to take her seriously. Indeed, it would be unsurprising if she takes unnecessary risks to make that happen.

The cold open of Hannibal (2001) makes for a worthy comparison. The audience remembers they loved the dialogue between Starling and Lecter, so it is clumsily mentioned. We remember Barney too, so he is shoehorned in as well. Lacking the confidence to introduce new signifiers to its universe, Red Dragon (2002) aside, the hollow, 'clip show' feel of Hannibal is a taste of the zero-calorie sequels to come in the next two decades.

The film is not perfect, and likely never was. Much has been written on the fairly transparent transphobia in Buffalo Bill's desire to wear a suit made out of women's skin, but the film then doubles down on its unflattering portrayal by trying to have it both ways. Starling tells the camera that "there's no correlation between transsexualism and violence," and Lecter (the film's psychoanalytic authority, remember) assures us that Buffalo Bill is "not a real transsexual" anyway. Yet despite those caveats, we are continually shown a TERFy cartoon of a man in a wig tucking his "precious" between his legs and an absurdly phallic gun. And, just we didn't quite get the message, a decent collection of Nazi memorabilia.

The film's director repeated the novel's contention that Buffalo Bill is not actually transgender, but someone so damaged that they are seeking some kind of transformation. This, for a brief moment, almost sounds true, and the film's deranged depiction of what it might be like to be transgender combined with its ambivalence feels distinctly disingenuous to me, especially given that — on an audience and Oscar-adjusted basis — Silence of the Lambs may very well be the most transphobic film to come out of Hollywood. Still, I remain torn on the death of the author, especially when I discover that Jonathan Demme went on to direct Philadelphia (1993), likely the most positive film about homophobia and HIV.


Nevertheless, as an adaption of Thomas Harris' original novel, the movie is almost flawless. The screenplay excises red herrings and tuns down the volume on some secondary characters. Crucially for the format, it amplifies Lecter's genius by not revealing that he knew everything all along and cuts Buffalo Bill's origin story for good measure too — good horror, after all, does not achieve its effect on the screen, but in the mind of the viewer. The added benefit of removing material from the original means that the film has time to slowly ratchet up the tension, and can remain patient and respectful of the viewer's intelligence throughout: it is, you could almost say, "Ready when you are, Sgt. Pembury". Otherwise, the film does not deviate too far from the original, taking the most liberty when it interleaves two narratives for the famous 'two doorbells' feint.

Dr. Lecter's upright stance when we meet him reminds me of the third act of Alfred Hitchcock's Notorious (1946), another picture freighted with meaningful stairs. Stanley Kubrick's The Killing (1956) began the now-shopworn trope of concealing a weapon in a flower box.

Two other points of deviation from the novel might be worthy of mention. In the book, a great deal is made of Dr. Lecter's penchant for Bach's Goldberg Variations, inducing a cultural resonance with other cinematic villains who have a taste for high art. It is also stressed in the book that it is the Canadian pianist Glenn Gould's recording too, although this is likely an attempt by Harris to demonstrate his own refined sensibilities — Lecter would surely have prefered a more historically-informed performance on the harpsichord. Yet it is glaringly obvious that it isn't Gould playing in the film at all; Gould's hypercanonical 1955 recording is faster and focused, whilst his 1981 release is much slower and contemplative. No doubt tedious issues around rights prevented the use of either recording, but I like to imagine that Gould himself nixed the idea.

The second change revolves around the film's most iconic quote. Deep underground, Dr. Lecter tries to spook Starling:

A census taker once tried to test me. I ate his liver with some fava beans and a nice Chianti.

The novel has this as "some fava beans and a big Amarone". No doubt the movie-going audience could not be trusted to know what an Amarone was, just as they were not to capable of recognising a philosopher. Nevertheless, substituting Chianti works better here as it cleverly foreshadows Tuscany (we discover that Lecter is living in Florence in the sequel), and it avoids the un-Lecterian tautology of 'big' — Amarone's, I am reliably informed, are big-bodied wines. Like Buffalo Bill's victims.

Yet that's not all. "The audience", according to TV Tropes:

... believe Lecter is merely confessing to one of his crimes. What most people would not know is that a common treatment for Lecter's "brand of crazy" is to use drugs of a class known as MAOIs (monoamine oxidase inhibitors). There are several things one must not eat when taking MAOIs, as they can case fatally low blood pressure, and as a physician and psychiatrist himself, Dr. Lecter would be well aware of this. These things include liver, fava beans, and red wine. In short, Lecter was telling Clarice that he was off his medication.

I could write more, but as they say, I'm having an old friend for dinner. The starling may be a common bird, but The Silence of the Lambs is that extremely rara avis indeed — the film that's better than the book. Ta ta...

14 February, 2021 06:19PM

hackergotchi for Steinar H. Gunderson

Steinar H. Gunderson

plocate 1.1.4 released

I made a minor release of plocate; as usual, https://plocate.sesse.net/ has the tarballs and such. The changelog reads:

plocate 1.1.4, February 14th, 2021

  - updatedb now uses ~15% less CPU time.

  - Installs a file CACHEDIR.tag into /var/lib/plocate, to mark the directory
    as autogenerated. Suggested by Marco d'Itri.

  - Manpage fixes; patch by Jakub Wilk.

The CPU time is, as usual, nothing really clever, but just a bunch of 1% optimizations. The plocate database is 0.1% larger or so, but it shouldn't really be noticed. There isn't io_uring support for updatedb yet, simply because I haven't bothered (it runs from cron/systemd anyway).

Also, upgrading libzstd from stable to unstable will make updatedb a few percent faster yet :-)

14 February, 2021 03:18PM

hackergotchi for Bits from Debian

Bits from Debian

I love Free Software Day 2021: Show your love for Free Software

ILoveFS banner

On this day February 14th, Debian joins the Free Software Foundation Europe in celebration of "I Love Free Software" day. This day takes the time to appreciate and applaud all those who contribute to the many areas of Free Software.

Debian sends all of our love and a giant “Thank you” to the upstream and downstream creators and maintainers, hosting providers, partners, and of course all of the Debian Developers and Contributors.

Thank you for all that you do in making Debian truly the Universal Operating System and for keeping and making Free Software Free!

Send some love and show some appreciation for Free Software by spreading the message and appreciation around the world, if you share in social media the hashtag used is: #ilovefs.

14 February, 2021 10:01AM by Donald Norwood

hackergotchi for Steinar H. Gunderson

Steinar H. Gunderson

Idle language musings

PHP makes the easy things easy, and in the process makes the wrong things easy and the right things hard.

C makes the easy things hard, the hard things possible, and in the process makes the wrong things just as easy as the right things.

Rust makes everything hard, but the wrong things even harder.

14 February, 2021 09:17AM

François Marier

Creating a Kodia media PC using a Raspberry Pi 4

Here's how I set up a media PC using Kodi (formerly XMBC) and a Raspberry Pi 4.


The hardware is fairly straightforward, but here's what I ended up getting:

You'll probably want to add a remote control to that setup. I used an old Streamzap I had lying around.

Installing the OS on the SD-card

Plug the SD card into a computer using a USB adapter.

Download the imager and use it to install Raspbian on the SDcard.

Then you can simply plug the SD card into the Pi and boot.

System configuration

Using sudo raspi-config, I changed the following:

  • Set hostname (System Options)
  • Wait for network at boot (System Options): needed for NFS
  • Disable screen blanking (Display Options)
  • Enable ssh (Interface Options)
  • Configure locale, timezone and keyboard (Localisation Options)
  • Set WiFi country (Localisation Options)

Then I enabled automatic updates:

apt install unattended-upgrades anacron

echo 'Unattended-Upgrade::Origins-Pattern {
        "origin=Raspberry Pi Foundation,codename=${distro_codename},label=Raspberry Pi Foundation";
};' | sudo tee /etc/apt/apt.conf.d/51unattended-upgrades-raspbian

Headless setup

Should you need to do the setup without a monitor, you can enable ssh by inserting the SD card into a computer and then creating an empty file called ssh in the boot partition.

Plug it into your router and boot it up. Check the IP that it received by looking at the active DHCP leases in your router's admin panel.

Then login:

ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no pi@192.168.1.xxx

using the default password of raspberry.


In order to secure the Pi, I followed most of the steps I usually take when setting up a new Linux server.

I created a new user account for admin and ssh access:

adduser francois
addgroup sshuser
adduser francois sshuser
adduser francois sudo

and changed the pi user password to a random one:

pwgen -sy 32
sudo passwd pi

before removing its admin permissions:

deluser pi adm
deluser pi sudo
deluser pi dialout
deluser pi cdrom
deluser pi lpadmin

Finally, I enabled the Uncomplicated Firewall by installing its package:

apt install ufw

and only allowing ssh connections.

After starting ufw using systemctl start ufw.service, you can check that it's configured as expected using ufw status. It should display the following:

Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)

Installing Kodi

Kodi is very straightforward to install since it's now part of the Raspbian repositories:

apt install kodi

To make it start at boot/login, while still being able to exit and use other apps if needed:

cp /etc/xdg/lxsession/LXDE-pi/autostart ~/.config/lxsession/LXDE-pi/
echo "@kodi" >> ~/.config/lxsession/LXDE-pi/autostart

Network File System

In order to avoid having to have all media storage connected directly to the Pi via USB, I setup an NFS share over my local network.

First, give static IP allocations to the server and the Pi in your DHCP server, then add it to the /etc/hosts file on your NFS server:    pi

Install the NFS server package:

apt instal nfs-kernel-server

Setup the directories to share in /etc/exports:

/pub/movies    pi(ro,insecure,all_squash,subtree_check)
/pub/tv_shows  pi(ro,insecure,all_squash,subtree_check)

Open the right ports on your firewall by putting this in /etc/network/iptables.up.rules:

-A INPUT -s -p udp -j ACCEPT
-A INPUT -s -p tcp --dport 111 -j ACCEPT
-A INPUT -s -p udp --dport 111 -j ACCEPT
-A INPUT -s -p udp --dport 123 -j ACCEPT
-A INPUT -s -p tcp --dport 600:1124 -j ACCEPT
-A INPUT -s -p udp --dport 600:1124 -j ACCEPT
-A INPUT -s -p tcp --dport 2049 -j ACCEPT
-A INPUT -s -p udp --dport 2049 -j ACCEPT

Finally, apply all of these changes:

systemctl restart nfs-kernel-server.service

On the Pi, put the server's static IP in /etc/hosts:    fileserver

and this in /etc/fstab:

fileserver:/data/movies  /kodi/movies  nfs  ro,bg,hard,noatime,async,nolock  0  0
fileserver:/data/tv      /kodi/tv      nfs  ro,bg,hard,noatime,async,nolock  0  0

Then create the mount points and mount everything:

mkdir -p /kodi/movies
mkdir /kodi/tv
mount /kodi/movies
mount /kodi/tv

14 February, 2021 03:26AM

February 13, 2021

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

RcppFastFloat 0.0.2: New Function

The second release of RcppFastFloat is now on CRAN. The package wraps fastfloat, another nice library by Daniel Lemire who showed in a recent arXiv paper that one can convert character representations of ‘numbers’ into floating point at rates at or exceeding one gigabyte per second.

Thanks to Brendan, this release adds a helper function as.double2() modeled after the base R function but using, of course, the features from fast_float in RcppFastFloat.

Release notes follow.

Changes in version 0.0.2 (2021-02-13)

  • New function as.double2() demonstrating fast_float (Brendan in #1)

Courtesy of my CRANberries, there is also a diffstat report for this release.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

13 February, 2021 09:53PM

Molly de Blanc

Proprietary (definition) – 02

I’ve had some good conversations about this attempt to define proprietary software. In many of these conversations, people focused on explicitly what I’m trying to not do (i.e. define “proprietary” by saying it’s not FOSS). Some people helped me clarify that I’m looking to do really, which is have a pithy way to explain proprietary to people who are never going to look at source code or pay someone to write new code for them. How do you explain to people who don’t care about technical matters nor have the language to discuss them? How do you talk about licenses to people who may not have the language for it? (In a past life I explained Creative Commons licenses to academics and educators.)

Talking about licensing seemed very important to people, as licenses are what define freedoms, restrictions, and restrictions that protect freedoms. With these points in mind, I present the following:

Proprietary software is software that comes with restrictions that retain control of how software can be used, shared, and changed through the use of copyright and licensing.

I worry that this is “too technical” and then I worry that I’m worrying too much about that. In this I added a truncated version of a common explanation of the Four Freedoms (typically use, study, modify, share). This is in part because I believe “study” is included in “modify.”

I included “copyright and licensing” in hopes that a reader would understand at least one of them. I also wanted to take into account that communities may have other policies (e.g. community guidelines) that might in some way restrict how software is used, shared, and changed. I don’t like “retain control” as a phrase, but it was suggested to me (thanks! If you want credit, just ping me). I think it’s pretty clear about the intention and consequence of proprietary licensing.

A potential criticism I see is that it’s not clear enough that you must be able to do all three (use, share, and change) in order for software to be FOSS and that restrictions on any of them renders software proprietary.

13 February, 2021 04:35PM by mollydb

Debian Community News

Nicolas Dandrimont, Pauline (or Maria) Climent-Pommeret & Debian, Outreachy, GSoC Conflict of Interest policy scandals

The Fellowship has recently blogged about FSFE President Matthias Kirschner misusing screengrabs from video calls with interns.

We hate to see interns names and photos brought into these disputes. The Debian Project Leader has been asked to cease vendettas against other volunteers. He continues the vendettas, he continues to let rumours hang over the heads of innocent volunteers so we have no choice other than showing who is really guilty.

Nicolas Dandrimont (olasd), an administrator in both Google Summer of Code (GSoC) and Outreachy, sent the email below, announcing that his girlfriend would apply for an internship. Dandrimont indicates he would still lurk around the administration of the program. He takes a swipe at the RTC projects managed by another volunteer. Incidentally, those RTC projects have been essential for many Free Software communities to work remotely during the pandemic. This is the email from Dandrimont:

Subject: Recusing myself from Outreachy applicant selection decisions, internships funding
Date: Fri, 14 Oct 2016 12:37:46 +0200
From: Nicolas Dandrimont <olasd@debian.org>
To: leader@debian.org, outreach@debian.org
CC: [mentors, redacted]

Hey all,

As of today, the person I'm involved with, Pauline Pommeret, is applying to an Outreachy internship in Debian (on the GPG cleanroom environment project - I don't see her mail on the list archive yet, so something must have gone wrong, but it should arrive soon enough).

To avoid an obvious conflict of interest, I am recusing myself for any decisions regarding applicant selections for this round.

I am of course still happy to serve as a liaison with the Outreachy program administrators, and to forward our applicants to them for general funding when selected, if the money allocated by Debian runs out.

This would especially be relevant, in my opinion, to RTC projects, as I'm not sure at all that we should fund them from Debian money directly. Karen Sandler also told me that one of the Outreachy sponsors was interested in funding interns on Reproducible Builds. All in all, we should be able to have two or three internship slots with Debian only disbursing one.

I'll stay on the outreach@d.o alias for now, but let me know if you need help ranking applicants, and I'll ask DSA to remove me so you can discuss at ease.

Nicolas Dandrimont


An application was commenced by this woman using the name Pauline Pommeret. Investigating, we find that she also uses other names, such as Maria Climent-Pommeret and chopopope. Some people have used aliases like this when doing something wrong in Debian but we don't want to jump to conclusions. It might be because she saw how people are subject to doxing in Debian. Some people don't use their real names to avoid consequences for their mistakes. The public shamings and Maria / Pauline's decision to use an alias could be a hint about why so few women volunteer in Debian.

As this woman hoping to become an intern began discussion with the Debian mentors, Dandrimont participated:

Subject: Re: Details concerning my application to the Outreachy program
Date: Sat, 15 Oct 2016 17:15:43 +0200
From: nicolas@dandrimont.eu
To: [redacted mentors], Pauline Pommeret , olasd@debian.org

(on mobile, sorry for the short reply) Just a detail, the contribution can happen/be merged after the application deadline too, no need to rush it. It only needs to happen before the selection, and the earlier the better, of course.

Le 15 octobre 2016 17:01:55 GMT+02:00, [mentor] a écrit :

On 15/10/16 15:37, [mentor] wrote:

Hi Pauline,

I don't know if ...

A few days later, on 18 October 2016, Dandrimont advised mentors that Pauline would withdraw from the selection process. At this stage, for privacy reasons, we decline to publish the withdrawal email.

In 2017, Debian did not participate in GSoC

Nicolas Dandrimont, olasd, Debian

In August 2017, Dandrimont resigned from the role of administrator. He did not give any forced public confession for the situation with his girlfriend. Dandrimont was not removed from the Debian keyring. He was not subjected to doxing in LWN and other places.

Some people in Debian appear to have immunity, like the leaders of communist regimes in Eastern Europe. Other people have been subject to severe punishments for the smallest mistakes.


In 2018, a similar situation occurred with other contributors to Debian in Google Summer of Code (GSoC). By way of precedent, mentors handled the case in the same way that Dandrimont had handled the case with his girlfriend.

In the 2018 GSoC case, all admins and mentors were made aware of the conflict of interest. Nobody kept it secret. Everybody in Debian knew. When the manager at Google, Stephanie Taylor, found out, she had a fit. It was Friday the 13th.

Subject: Concerns around Debian GSoC students and conflict of interest
Date: Fri, 13 Jul 2018 UTC
From: Stephanie Taylor <sttaylor@google.com>
To: Pranav Jain <contact@pranavjain.me>, prabaharan jaminy , Alexander Wirt , [redacted], molly.deblanc@gmail.com

Hello Debian Org Admins,

It has come to our attention that one of your [name and title redacted], is the [relationship redacted] of one of your students - [student redacted], which is in clear violation of the GSoC Rules both [redacted] and [redacted] agreed to. We will have to remove [redacted] from the program immediately.

One of the Debian people explained to Taylor the precedent set by Nicolas Dandrimont and Pauline (or Maria) Climent-Pommeret in Outreachy 2016. Taylor sent additional comments

Stephanie Taylor (Google): It is never okay to have a conflict of interest like you had with [redacted/leadership role], having a [redacted/relationships], etc. in the program as a student. That is a clear conflict of interest that would influence the mentor whether or not they intend it to. No one wants to fail the [redacted/leadership role]'s [redacted/relationship] during the program, there is already favoritism even if the [redacted/leadership role] is not involved in the student's selection or mentoring.

The volunteers who were blackmailed, removed from the Debian keyring and subject to sustained public attacks that continue to this day were not involved in any romantic conflict of interest. It looks like Debian uses people as scapegoats so that other people can get away with anything.

Matthias Kirschner, FSFE, Objectifying female interns

13 February, 2021 10:30AM

February 12, 2021

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

RcppSimdJson 0.1.4 on CRAN: New Improvements

Brendan and I are happy to share that a new RcppSimdJson release 0.1.4 arrived on CRAN earlier today. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire at QCon (also voted best talk).

This version brings a new option to always return list types, tweaks to setting option in the the request and other small improvements. The NEWS entry follows.

Changes in version 0.1.4 (2021-02-12)

  • Support additional headers in fload (Dirk in #60).

  • Enable continuous integration via GitHub Actions using run.sh from r-ci repo (Dirk in #61, #62).

  • Add option to always return list to fparse()/fload() (Brendan in #65 closing #64).

Courtesy of my CRANberries, there is also a diffstat report for this release.

For questions, suggestions, or issues please use the issue tracker at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

12 February, 2021 10:13PM

Sylvain Beucler

Godot GDScript REPL

When experimenting with Godot and its GDScript language, I realized that I missed a good old REPL (Read-Eval-Print Loop) to familiarize myself with the language and API.

This is now possible with this new Godot Editor plugin :)

Try it at:

12 February, 2021 09:33AM

February 10, 2021

Free Software Fellowship

Matthias Kirschner & FSFE: Objectifying women for Valentines Day

For ordinary people, 14 February is simply Valentines Day.

Matthias Kirschner, FSFE president, has decided to intrude upon your date with an I Love Free Software campaign. At first we thought it was just quirky and awkward, like teenagers on a first date. Thinking about it more deeply, we find deeper revelations about diversity in free, open source software.

After Kirschner sacked all the women on permanent contracts in the FSFE office, his intrusion on Valentines Day is incredibly out of place, unless you feel that misogyny and submissive women are the goal.

FSFE published this screengrab from an online meeting:

Bonnie Mehring, FSFE, consent

Later, the following image was distributed. It looks like they cut the intern's face out of the screengrab and used it within a heart template:

Bonnie Mehring, FSFE, consent

Sweet? Maybe not. Did the intern know her photo would be used like this when doing a video call? Did she freely consent or did she feel an obligation to do it, like previous interns?

We couldn't find images of male interns used like this. We made one up:

matthias kirschner, fsfe, defamation, volunteers

Back to the deeper questions of diversity and open source culture.

In the leaked FSFE diversity strategy, it was revealed that Kirschner organizes volunteers to keep contacting women after their internship finished. He believes that they will keep working without pay. We take this as a sign of a personality cult and an ego problem. Kirschner believes he is the most important thing in the lives of these women, so much so that they will work for free, like members of a cult.

Why do other male members of the FSFE cult partake in this foolishness on Valentines Day? It may simply be because one of the biggest free software conferences is at the beginning of February. Kirschner ordered the interns to run around smiling at men and collecting photos.

fsfe, olga, fosdem

It is a money-go-round: money pays for interns, interns register more male donors on recurring subscriptions. Very little Free Software is developed at FSFE.

The deeper insight we take from these images is that German men, and FSFE is almost eighty percent German, have the same feelings for their spouse as they have for their gadgets or their beer.

fsfe, love beer

fsfe, love gadgets

valentines day

10 February, 2021 08:40AM

February 09, 2021

hackergotchi for Gunnar Wolf

Gunnar Wolf

And now, Bullseye images are also built for the RPi

Public service announcement

In case you want to run our latest release (still cooking, of course) in your Raspberries — I have enabled builds for both Debian 10 (Stable, Buster) and Debian 11 (Testing, Bullseye). Go grab it!

Oh… Yes, we are currently failing the builds of ARM64 (RPi3 and RPi4) ☹ Something due to python3-minimal unwilling to get installed right. But that should be fixed soon! Can you help us? Take a look at the [build log for RPi3, Bullseye](https://raspi.debian.net/daily/raspi_3_bullseye.log), or just focus on the step where it breaks It seems to have been fixed, woohoo!:

Setting up python3-minimal (3.9.1-1) ...

2021-02-09 08:56:38 DEBUG STDERR: E: Can not write log (Is /dev/pts mounted?) - posix_openpt (19: No such device)
Segmentation fault
dpkg: error processing package python3-minimal (--configure):
 installed python3-minimal package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
E: Sub-process /usr/bin/dpkg returned an error code (1)

Anyway, as you can see, the eight built images work fine and are tested, at least, for basic support!

09 February, 2021 05:00PM

Molly de Blanc

Proprietary (definition)

I recently had the occasion to try and find a definition of “proprietary” in terms of software that is not on Wikipedia. Most of the discussion on the issue I found was focused on what free and open source software is, and that anything that isn’t FOSS is proprietary. I don’t think the debate is as simple as this, especially if you want to get into conversations about nuance around things like Open Core.

The problem with defining proprietary software by what it isn’t, or at least that it isn’t FOSS, means that we cannot concisely communicate what makes something proprietary. Instead, we leave it up to the people we’re trying to communicate with to dig through a history of rhetoric, copyright law, and licensing in order to understand what it actually means for something to be FOSS, and what it means for something to be anything else. It is also just less satisfying, in my opinion, to define something by what it lacks rather than by what it is.

I’ll start by proposing the following definition:

Proprietary software is software that comes with restrictions on what users can do with the software and the source code that constitutes said software.

I think the most controversial part of this sentence is the wording “software that comes with restrictions.” In earlier attempts of this I wrote “software that restricts.” This sort of active wording, which I used for years in my capacity at work, is misleading. In the case of proprietary software, it is the licensing and laws around it that restrict what you can do. For software to restrict you, it must be that the way the software is being implemented or used restricts you.

To be clear, this is my first proposal. I look forward to discussing this further!

09 February, 2021 02:08PM by mollydb

hackergotchi for Kees Cook

Kees Cook

security things in Linux v5.8

Previously: v5.7

Linux v5.8 was released in August, 2020. Here’s my summary of various security things that caught my attention:

arm64 Branch Target Identification
Dave Martin added support for ARMv8.5’s Branch Target Instructions (BTI), which are enabled in userspace at execve() time, and all the time in the kernel (which required manually marking up a lot of non-C code, like assembly and JIT code).

With this in place, Jump-Oriented Programming (JOP, where code gadgets are chained together with jumps and calls) is no longer available to the attacker. An attacker’s code must make direct function calls. This basically reduces the “usable” code available to an attacker from every word in the kernel text to only function entries (or jump targets). This is a “low granularity” forward-edge Control Flow Integrity (CFI) feature, which is important (since it greatly reduces the potential targets that can be used in an attack) and cheap (implemented in hardware). It’s a good first step to strong CFI, but (as we’ve seen with things like CFG) it isn’t usually strong enough to stop a motivated attacker. “High granularity” CFI (which uses a more specific branch-target characteristic, like function prototypes, to track expected call sites) is not yet a hardware supported feature, but the software version will be coming in the future by way of Clang’s CFI implementation.

arm64 Shadow Call Stack
Sami Tolvanen landed the kernel implementation of Clang’s Shadow Call Stack (SCS), which protects the kernel against Return-Oriented Programming (ROP) attacks (where code gadgets are chained together with returns). This backward-edge CFI protection is implemented by keeping a second dedicated stack pointer register (x18) and keeping a copy of the return addresses stored in a separate “shadow stack”. In this way, manipulating the regular stack’s return addresses will have no effect. (And since a copy of the return address continues to live in the regular stack, no changes are needed for back trace dumps, etc.)

It’s worth noting that unlike BTI (which is hardware based), this is a software defense that relies on the location of the Shadow Stack (i.e. the value of x18) staying secret, since the memory could be written to directly. Intel’s hardware ROP defense (CET) uses a hardware shadow stack that isn’t directly writable. ARM’s hardware defense against ROP is PAC (which is actually designed as an arbitrary CFI defense — it can be used for forward-edge too), but that depends on having ARMv8.3 hardware. The expectation is that SCS will be used until PAC is available.

Kernel Concurrency Sanitizer infrastructure added
Marco Elver landed support for the Kernel Concurrency Sanitizer, which is a new debugging infrastructure to find data races in the kernel, via CONFIG_KCSAN. This immediately found real bugs, with some fixes having already landed too. For more details, see the KCSAN documentation.

new capabilities
Alexey Budankov added CAP_PERFMON, which is designed to allow access to perf(). The idea is that this capability gives a process access to only read aspects of the running kernel and system. No longer will access be needed through the much more powerful abilities of CAP_SYS_ADMIN, which has many ways to change kernel internals. This allows for a split between controls over the confidentiality (read access via CAP_PERFMON) of the kernel vs control over integrity (write access via CAP_SYS_ADMIN).

Alexei Starovoitov added CAP_BPF, which is designed to separate BPF access from the all-powerful CAP_SYS_ADMIN. It is designed to be used in combination with CAP_PERFMON for tracing-like activities and CAP_NET_ADMIN for networking-related activities. For things that could change kernel integrity (i.e. write access), CAP_SYS_ADMIN is still required.

network random number generator improvements
Willy Tarreau made the network code’s random number generator less predictable. This will further frustrate any attacker’s attempts to recover the state of the RNG externally, which might lead to the ability to hijack network sessions (by correctly guessing packet states).

fix various kernel address exposures to non-CAP_SYSLOG
I fixed several situations where kernel addresses were still being exposed to unprivileged (i.e. non-CAP_SYSLOG) users, though usually only through odd corner cases. After refactoring how capabilities were being checked for files in /sys and /proc, the kernel modules sections, kprobes, and BPF exposures got fixed. (Though in doing so, I briefly made things much worse before getting it properly fixed. Yikes!)

RISCV W^X detection
Following up on his recent work to enable strict kernel memory protections on RISCV, Zong Li has now added support for CONFIG_DEBUG_WX as seen for other architectures. Any writable and executable memory regions in the kernel (which are lovely targets for attackers) will be loudly noted at boot so they can get corrected.

execve() refactoring continues
Eric W. Biederman continued working on execve() refactoring, including getting rid of the frequently problematic recursion used to locate binary handlers. I used the opportunity to dust off some old binfmt_script regression tests and get them into the kernel selftests.

multiple /proc instances
Alexey Gladkov modernized /proc internals and provided a way to have multiple /proc instances mounted in the same PID namespace. This allows for having multiple views of /proc, with different features enabled. (Including the newly added hidepid=4 and subset=pid mount options.)

set_fs() removal continues
Christoph Hellwig, with Eric W. Biederman, Arnd Bergmann, and others, have been diligently working to entirely remove the kernel’s set_fs() interface, which has long been a source of security flaws due to weird confusions about which address space the kernel thought it should be accessing. Beyond things like the lower-level per-architecture signal handling code, this has needed to touch various parts of the ELF loader, and networking code too.

READ_IMPLIES_EXEC is no more for native 64-bit
The READ_IMPLIES_EXEC flag was a work-around for dealing with the addition of non-executable (NX) memory when x86_64 was introduced. It was designed as a way to mark a memory region as “well, since we don’t know if this memory region was expected to be executable, we must assume that if we need to read it, we need to be allowed to execute it too”. It was designed mostly for stack memory (where trampoline code might live), but it would carry over into all mmap() allocations, which would mean sometimes exposing a large attack surface to an attacker looking to find executable memory. While normally this didn’t cause problems on modern systems that correctly marked their ELF sections as NX, there were still some awkward corner-cases. I fixed this by splitting READ_IMPLIES_EXEC from the ELF PT_GNU_STACK marking on x86 and arm/arm64, and declaring that a native 64-bit process would never gain READ_IMPLIES_EXEC on x86_64 and arm64, which matches the behavior of other native 64-bit architectures that correctly didn’t ever implement READ_IMPLIES_EXEC in the first place.

array index bounds checking continues
As part of the ongoing work to use modern flexible arrays in the kernel, Gustavo A. R. Silva added the flex_array_size() helper (as a cousin to struct_size()). The zero/one-member into flex array conversions continue with over a hundred commits as we slowly get closer to being able to build with -Warray-bounds.

scnprintf() replacement continues
Chen Zhou joined Takashi Iwai in continuing to replace potentially unsafe uses of sprintf() with scnprintf(). Fixing all of these will make sure the kernel avoids nasty buffer concatenation surprises.

That’s it for now! Let me know if there is anything else you think I should mention here. Next up: Linux v5.9.

© 2021, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 License.
CC BY-SA 4.0

09 February, 2021 12:47AM by kees

February 07, 2021

Enrico Zini

hackergotchi for Chris Lamb

Chris Lamb

Favourite books of 2020

I won't reveal precisely how many books I read in 2020, but it was definitely an improvement on 74 in 2019, 53 in 2018 and 50 in 2017. But not only did I read more in a quantitative sense, the quality seemed higher as well. There were certainly fewer disappointments: given its cultural resonance, I was nonplussed by Nick Hornby's Fever Pitch and whilst Ian Fleming's The Man with the Golden Gun was a little thin (again, given the obvious influence of the Bond franchise) the booked lacked 'thinness' in a way that made it interesting to critique. The weakest novel I read this year was probably J. M. Berger's Optimal, but even this hybrid of Ready Player One late-period Black Mirror wasn't that cringeworthy, all things considered. Alas, graphic novels continue to not quite be my thing, I'm afraid.

I perhaps experienced more disappointments in the non-fiction section. Paul Bloom's Against Empathy was frustrating, particularly in that it expended unnecessary energy battling its misleading title and accepted terminology, and it could so easily have been an 20-minute video essay instead). (Elsewhere in the social sciences, David and Goliath will likely be the last Malcolm Gladwell book I voluntarily read.) After so many positive citations, I was also more than a little underwhelmed by Shoshana Zuboff's The Age of Surveillance Capitalism, and after Ryan Holiday's many engaging reboots of Stoic philosophy, his Conspiracy (on Peter Thiel and Hulk Hogan taking on Gawker) was slightly wide of the mark for me.

Anyway, here follows a selection of my favourites from 2020, in no particular order:



Wolf Hall & Bring Up the Bodies & The Mirror and the Light

Hilary Mantel

During the early weeks of 2020, I re-read the first two parts of Hilary Mantel's Thomas Cromwell trilogy in time for the March release of The Mirror and the Light. I had actually spent the last few years eagerly following any news of the final instalment, feigning outrage whenever Mantel appeared to be spending time on other projects.

Wolf Hall turned out to be an even better book than I remembered, and when The Mirror and the Light finally landed at midnight on 5th March, I began in earnest the next morning. Note that date carefully; this was early 2020, and the book swiftly became something of a heavy-handed allegory about the world at the time. That is to say — and without claiming that I am Monsieur Cromuel in any meaningful sense — it was an uneasy experience to be reading about a man whose confident grasp on his world, friends and life was slipping beyond his control, and at least in Cromwell's case, was heading inexorably towards its denouement.

The final instalment in Mantel's trilogy is not perfect, and despite my love of her writing I would concur with the judges who decided against awarding her a third Booker Prize. For instance, there is something of the longueur that readers dislike in the second novel, although this might not be entirely Mantel's fault — after all, the rise of the "ugly" Anne of Cleves and laborious trade negotiations for an uninspiring mineral (this is no Herbertian 'spice') will never match the court intrigues of Anne Boleyn, Jane Seymour and that man for all seasons, Thomas More. Still, I am already looking forward to returning to the verbal sparring between King Henry and Cromwell when I read the entire trilogy once again, tentatively planned for 2022.


The Fault in Our Stars

John Green

I came across John Green's The Fault in Our Stars via a fantastic video by Lindsay Ellis discussing Roland Barthes famous 1967 essay on authorial intent. However, I might have eventually come across The Fault in Our Stars regardless, not because of Green's status as an internet celebrity of sorts but because I'm a complete sucker for this kind of emotionally-manipulative bildungsroman, likely due to reading Philip Pullman's His Dark Materials a few too many times in my teens.

Although its title is taken from Shakespeare's Julius Caesar, The Fault in Our Stars is actually more Romeo & Juliet. Hazel, a 16-year-old cancer patient falls in love with Gus, an equally ill teen from her cancer support group. Hazel and Gus share the same acerbic (and distinctly unteenage) wit and a love of books, centred around Hazel's obsession of An Imperial Affliction, a novel by the meta-fictional author Peter Van Houten. Through a kind of American version of Jim'll Fix It, Gus and Hazel go and visit Van Houten in Amsterdam.

I'm afraid it's even cheesier than I'm describing it. Yet just as there is a time and a place for Michelin stars and Haribo Starmix, there's surely a place for this kind of well-constructed but altogether maudlin literature. One test for emotionally manipulative works like this is how well it can mask its internal contradictions — while Green's story focuses on the universalities of love, fate and the shortness of life (as do almost all of his works, it seems), The Fault in Our Stars manages to hide, for example, that this is an exceedingly favourable treatment of terminal illness that is only possible for the better off. The 2014 film adaptation does somewhat worse in peddling this fantasy (and has a much weaker treatment of the relationship between the teens' parents too, an underappreciated subtlety of the book).

The novel, however, is pretty slick stuff, and it is difficult to fault it for what it is. For some comparison, I later read Green's Looking for Alaska and Paper Towns which, as I mention, tug at many of the same strings, but they don't come together nearly as well as The Fault in Our Stars. James Joyce claimed that "sentimentality is unearned emotion", and in this respect, The Fault in Our Stars really does earn it.


The Plague

Albert Camus

P. D. James' The Children of Men, George Orwell's Nineteen Eighty-Four, Arthur Koestler's Darkness at Noon ... dystopian fiction was already a theme of my reading in 2020, so given world events it was an inevitability that I would end up with Camus's novel about a plague that swept through the Algerian city of Oran.

Is The Plague an allegory about the Nazi occupation of France during World War Two? Where are all the female characters? Where are the Arab ones? Since its original publication in 1947, there's been so much written about The Plague that it's hard to say anything new today. Nevertheless, I was taken aback by how well it captured so much of the nuance of 2020. Whilst we were saying just how 'unprecedented' these times were, it was eerie how a novel written in the 1940s could accurately how many of us were feeling well over seventy years on later: the attitudes of the people; the confident declarations from the institutions; the misaligned conversations that led to accidental misunderstandings. The disconnected lovers.

The only thing that perhaps did not work for me in The Plague was the 'character' of the church. Although I could appreciate most of the allusion and metaphor, it was difficult for me to relate to the significance of Father Paneloux, particularly regarding his change of view on the doctrinal implications of the virus, and — spoiler alert — that he finally died of a "doubtful case" of the disease, beyond the idea that Paneloux's beliefs are in themselves "doubtful". Answers on a postcard, perhaps.

The Plague even seemed to predict how we, at least speaking of the UK, would react when the waves of the virus waxed and waned as well:

The disease stiffened and carried off three or four patients who were expected to recover. These were the unfortunates of the plague, those whom it killed when hope was high

It somehow captured the nostalgic yearning for high-definition videos of cities and public transport; one character even visits the completely deserted railway station in Oman simply to read the timetables on the wall.


Tinker, Tailor, Soldier, Spy

John le Carré

There's absolutely none of the Mad Men glamour of James Bond in John le Carré's icy world of Cold War spies:

Small, podgy, and at best middle-aged, Smiley was by appearance one of London's meek who do not inherit the earth. His legs were short, his gait anything but agile, his dress costly, ill-fitting, and extremely wet.

Almost a direct rebuttal to Ian Fleming's 007, Tinker, Tailor has broken-down cars, bad clothes, women with their own internal and external lives (!), pathetically primitive gadgets, and (contra Mad Men) hangovers that significantly longer than ten minutes. In fact, the main aspect that the mostly excellent 2011 film adaption doesn't really capture is the smoggy and run-down nature of 1970s London — this is not your proto-Cool Britannia of Austin Powers or GTA:1969, the city is truly 'gritty' in the sense there is a thin film of dirt and grime on every surface imaginable.

Another angle that the film cannot capture well is just how purposefully the novel does not mention the United States. Despite the US obviously being the dominant power, the British vacillate between pretending it doesn't exist or implying its irrelevance to the matter at hand. This is no mistake on Le Carré's part, as careful readers are rewarded by finding this denial of US hegemony in metaphor throughout --pace Ian Fleming, there is no obvious Felix Leiter to loudly throw money at the problem or a Sheriff Pepper to serve as cartoon racist for the Brits to feel superior about. By contrast, I recall that a clever allusion to "dusty teabags" is subtly mirrored a few paragraphs later with a reference to the installation of a coffee machine in the office, likely symbolic of the omnipresent and unavoidable influence of America. (The officer class convince themselves that coffee is a European import.) Indeed, Le Carré communicates a feeling of being surrounded on all sides by the peeling wallpaper of Empire.

Oftentimes, the writing style matches the graceless and inelegance of the world it depicts. The sentences are dense and you find your brain performing a fair amount of mid-flight sentence reconstruction, reparsing clauses, commas and conjunctions to interpret Le Carré's intended meaning. In fact, in his eulogy-cum-analysis of Le Carré's writing style, William Boyd, himself a ventrioquilist of Ian Fleming, named this intentional technique 'staccato'. Like the musical term, I suspect the effect of this literary staccato is as much about the impact it makes on a sentence as the imperceptible space it generates after it.

Lastly, the large cast in this sprawling novel is completely believable, all the way from the Russian spymaster Karla to minor schoolboy Roach — the latter possibly a stand-in for Le Carré himself. I got through the 500-odd pages in just a few days, somehow managing to hold the almost-absurdly complicated plot in my head. This is one of those classic books of the genre that made me wonder why I had not got around to it before.


The Nickel Boys

Colson Whitehead

According to the judges who awarded it the Pulitzer Prize for Fiction, The Nickel Boys is "a devastating exploration of abuse at a reform school in Jim Crow-era Florida" that serves as a "powerful tale of human perseverance, dignity and redemption". But whilst there is plenty of this perseverance and dignity on display, I found little redemption in this deeply cynical novel.

It could almost be read as a follow-up book to Whitehead's popular The Underground Railroad, which itself won the Pulitzer Prize in 2017. Indeed, each book focuses on a young protagonist who might be euphemistically referred to as 'downtrodden'. But The Nickel Boys is not only far darker in tone, it feels much closer and more connected to us today. Perhaps this is unsurprising, given that it is based on the story of the Dozier School in northern Florida which operated for over a century before its long history of institutional abuse and racism was exposed a 2012 investigation. Nevertheless, if you liked the social commentary in The Underground Railroad, then there is much more of that in The Nickel Boys:

Perhaps his life might have veered elsewhere if the US government had opened the country to colored advancement like they opened the army. But it was one thing to allow someone to kill for you and another to let him live next door.

Sardonic aperçus of this kind are pretty relentless throughout the book, but it never tips its hand too far into on nihilism, especially when some of the visual metaphors are often first-rate: "An American flag sighed on a pole" is one I can easily recall from memory. In general though, The Nickel Boys is not only more world-weary in tenor than his previous novel, the United States it describes seems almost too beaten down to have the energy conjure up the Swiftian magical realism that prevented The Underground Railroad from being overly lachrymose. Indeed, even we Whitehead transports us a present-day New York City, we can't indulge in another kind of fantasy, the one where America has solved its problems:

The Daily News review described the [Manhattan restaurant] as nouveau Southern, "down-home plates with a twist." What was the twist — that it was soul food made by white people?

It might be overly reductionist to connect Whitehead's tonal downshift with the racial justice movements of the past few years, but whatever the reason, we've ended up with a hard-hitting, crushing and frankly excellent book.


True Grit & No Country for Old Men

Charles Portis & Cormac McCarthy

It's one of the most tedious cliches to claim the book is better than the film, but these two books are of such high quality that even the Coen Brothers at their best cannot transcend them. I'm grouping these books together here though, not because their respective adaptations will exemplify some of the best cinema of the 21st century, but because of their superb treatment of language.

Take the use of dialogue. Cormac McCarthy famously does not use any punctuation — "I believe in periods, in capitals, in the occasional comma, and that's it" — but the conversations in No Country for Old Men together feel familiar and commonplace, despite being relayed through this unconventional technique. In lesser hands, McCarthy's written-out Texan drawl would be the novelistic equivalent of white rap or Jar Jar Binks, but not only is the effect entirely gripping, it helps you to believe you are physically present in the many intimate and domestic conversations that hold this book together. Perhaps the cinematic familiarity helps, as you can almost hear Tommy Lee Jones' voice as Sheriff Bell from the opening page to the last.

Charles Portis' True Grit excels in its dialogue too, but in this book it is not so much in how it flows (although that is delightful in its own way) but in how forthright and sardonic Maddie Ross is:

"Earlier tonight I gave some thought to stealing a kiss from you, though you are very young, and sick and unattractive to boot, but now I am of a mind to give you five or six good licks with my belt."

"One would be as unpleasant as the other."

Perhaps this should be unsurprising. Maddie, a fourteen-year-old girl from Yell County, Arkansas, can barely fire her father's heavy pistol, so she can only has words to wield as her weapon. Anyway, it's not just me who treasures this book. In her encomium that presages most modern editions, Donna Tartt of The Secret History fame traces the novels origins through Huckleberry Finn, praising its elegance and economy: "The plot of True Grit is uncomplicated and as pure in its way as one of the Canterbury Tales". I've read any Chaucer, but I am inclined to agree.

Tartt also recalls that True Grit vanished almost entirely from the public eye after the release of John Wayne's flimsy cinematic vehicle in 1969 — this earlier film was, Tartt believes, "good enough, but doesn't do the book justice". As it happens, reading a book with its big screen adaptation as a chaser has been a minor theme of my 2020, including P. D. James' The Children of Men, Kazuo Ishiguro's Never Let Me Go, Patricia Highsmith's Strangers on a Train, James Ellroy's The Black Dahlia, John Green's The Fault in Our Stars, John le Carré's Tinker, Tailor Soldier, Spy and even a staged production of Charles Dicken's A Christmas Carol streamed from The Old Vic. For an autodidact with no academic background in literature or cinema, I've been finding this an effective and enjoyable means of getting closer to these fine books and films — it is precisely where they deviate (or perhaps where they are deficient) that offers a means by which one can see how they were constructed. I've also found that adaptations can also tell you a lot about the culture in which they were made: take the 'straightwashing' in the film version of Strangers on a Train (1951) compared to the original novel, for example. It is certainly true that adaptions rarely (as Tartt put it) "do the book justice", but she might be also right to alight on a legal metaphor, for as the saying goes, to judge a movie in comparison to the book is to do both a disservice.


The Glass Hotel

Emily St. John Mandel

In The Glass Hotel, Mandel somehow pulls off the impossible; writing a loose roman-à-clef on Bernie Madoff, a Ponzi scheme and the ephemeral nature of finance capital that is tranquil and shimmeringly beautiful. Indeed, don't get the wrong idea about the subject matter; this is no over over-caffeinated The Big Short, as The Glass Hotel is less about a Madoff or coked-up financebros but the fragile unreality of the late 2010s, a time which was, as we indeed discovered in 2020, one event away from almost shattering completely.

Mandel's prose has that translucent, phantom quality to it where the chapters slip through your fingers when you try to grasp at them, and the plot is like a ghost ship that that slips silently, like the Mary Celeste, onto the Canadian water next to which the eponymous 'Glass Hotel' resides. Indeed, not unlike The Overlook Hotel, the novel so overflows with symbolism so that even the title needs to evoke the idea of impermanence — permanently living in a hotel might serve as a house, but it won't provide a home. It's risky to generalise about such things post-2016, but the whole story sits in that the infinitesimally small distance between perception and reality, a self-constructed culture that is not so much 'post truth' but between them.

There's something to consider in almost every character too. Take the stand-in for Bernie Madoff: no caricature of Wall Street out of a 1920s political cartoon or Brechtian satire, Jonathan Alkaitis has none of the oleaginous sleaze of a Dominic Strauss-Kahn, the cold sociopathy of a Marcus Halberstam nor the well-exercised sinuses of, say, Jordan Belford. Alkaitis is — dare I say it? — eminently likeable, and the book is all the better for it. Even the C-level characters have something to say: Enrico, trivially escaping from the regulators (who are pathetically late to the fraud without Mandel ever telling us explicitly), is daydreaming about the girlfriend he abandoned in New York: "He wished he'd realised he loved her before he left". What was in his previous life that prevented him from doing so? Perhaps he was never in love at all, or is love itself just as transient as the imaginary money in all those bank accounts? Maybe he fell in love just as he crossed safely into Mexico? When, precisely, do we fall in love anyway?

I went on to read Mandel's Last Night in Montreal, an early work where you can feel her reaching for that other-worldly quality that she so masterfully achieves in The Glass Hotel. Her fêted Station Eleven is on my must-read list for 2021. "What is truth?" asked Pontius Pilate. Not even Mandel cannot give us the answer, but this will certainly do for now.


Running the Light

Sam Tallent

Although it trades in all of the clichés and stereotypes of the stand-up comedian (the triumvirate of drink, drugs and divorce), Sam Tallent's debut novel depicts an extremely convincing fictional account of a touring road comic.

The comedian Doug Stanhope (who himself released a fairly decent No Encore for the Donkey memoir in 2020) hyped Sam's book relentlessly on his podcast during lockdown... and justifiably so. I ripped through Running the Light in a few short hours, the only disappointment being that I can't seem to find videos online of Sam that come anywhere close to match up to his writing style. If you liked the rollercoaster energy of Paul Beatty's The Sellout, the cynicism of George Carlin and the car-crash invertibility of final season Breaking Bad, check this great book out.



Inside Story

Martin Amis

This was my first introduction to Martin Amis's work after hearing that his "novelised autobiography" contained a fair amount about Christopher Hitchens, an author with whom I had a one of those rather clichéd parasocial relationship with in the early days of YouTube. (Hey, it could have been much worse.) Amis calls his book a "novelised autobiography", and just as much has been made of its quasi-fictional nature as the many diversions into didactic writing advice that betwixt each chapter: "Not content with being a novel, this book also wants to tell you how to write novels", complained Tim Adams in The Guardian.

I suspect that reviewers who grew up with Martin since his debut book in 1973 rolled their eyes at yet another demonstration of his manifest cleverness, but as my first exposure to Amis's gift of observation, I confess that I was thought it was actually kinda clever. Try, for example, "it remains a maddening truth that both sexual success and sexual failure are steeply self-perpetuating" or "a hospital gym is a contradiction – like a young Conservative", etc. Then again, perhaps I was experiencing a form of nostalgia for a pre-Gamergate YouTube, when everything in the world was a lot simpler... or at least things could be solved by articulate gentlemen who honed their art of rhetoric at the Oxford Union.

I went on to read Martin's first novel, The Rachel Papers (is it 'arrogance' if you are, indeed, that confident?), as well as his 1997 Night Train. I plan to read more of him in the future.


The Collected Essays, Journalism and Letters: Volume 1 & Volume 2 & Volume 3 & Volume 4

George Orwell

These deceptively bulky four volumes contain all of George Orwell's essays, reviews and correspondence, from his teenage letters sent to local newspapers to notes to his literary executor on his deathbed in 1950. Reading this was part of a larger, multi-year project of mine to cover the entirety of his output.

By including this here, however, I'm not recommending that you read everything that came out of Orwell's typewriter. The letters to friends and publishers will only be interesting to biographers or hardcore fans (although I would recommend Dorian Lynskey's The Ministry of Truth: A Biography of George Orwell's 1984 first). Furthermore, many of his book reviews will be of little interest today. Still, some insights can be gleaned; if there is any inconsistency in this huge corpus is that his best work is almost 'too' good and too impactful, making his merely-average writing appear like hackwork. There are some gems that don't make the usual essay collections too, and some of Orwell's most astute social commentary came out of series of articles he wrote for the left-leaning newspaper Tribune, related in many ways to the US Jacobin. You can also see some of his most famous ideas start to take shape years — if not decades — before they appear in his novels in these prototype blog posts.

I also read Dennis Glover's novelised account of the writing of Nineteen-Eighty Four called The Last Man in Europe, and I plan to re-read some of Orwell's earlier novels during 2021 too, including A Clergyman's Daughter and his 'antebellum' Coming Up for Air that he wrote just before the Second World War; his most under-rated novel in my estimation. As it happens, and with the exception of the US and Spain, copyright in the works published in his lifetime ends on 1st January 2021. Make of that what you will.


Capitalist Realism & Chavs: The Demonisation of the Working Class

Mark Fisher & Owen Jones

These two books are not natural companions to one another and there is likely much that Jones and Fisher would vehemently disagree on, but I am pairing these books together here because they represent the best of the 'political' books I read in 2020.

Mark Fisher was a dedicated leftist whose first book, Capitalist Realism, marked an important contribution to political philosophy in the UK. However, since his suicide in early 2017, the currency of his writing has markedly risen, and Fisher is now frequently referenced due to his belief that the prevalence of mental health conditions in modern life is a side-effect of various material conditions, rather than a natural or unalterable fact "like weather". (Of course, our 'weather' is being increasingly determined by a combination of politics, economics and petrochemistry than pure randomness.) Still, Fisher wrote on all manner of topics, from the 2012 London Olympics and "weird and eerie" electronic music that yearns for a lost future that will never arrive, possibly prefiguring or influencing the Fallout video game series.

Saying that, I suspect Fisher will resonate better with a UK audience more than one across the Atlantic, not necessarily because he was minded to write about the parochial politics and culture of Britain, but because his writing often carries some exasperation at the suppression of class in favour of identity-oriented politics, a viewpoint not entirely prevalent in the United States outside of, say, Touré F. Reed or the late Michael Brooks. (Indeed, Fisher is likely best known in the US as the author of his controversial 2013 essay, Exiting the Vampire Castle, but that does not figure greatly in this book). Regardless, Capitalist Realism is an insightful, damning and deeply unoptimistic book, best enjoyed in the warm sunshine — I found it an ironic compliment that I had quoted so many paragraphs that my Kindle's copy protection routines prevented me from clipping any further.

Owen Jones needs no introduction to anyone who regularly reads a British newspaper, especially since 2015 where he unofficially served as a proxy and punching bag for expressing frustrations with the then-Labour leader, Jeremy Corbyn. However, as the subtitle of Jones' 2012 book suggests, Chavs attempts to reveal the "demonisation of the working class" in post-financial crisis Britain. Indeed, the timing of the book is central to Jones' analysis, specifically that the stereotype of the "chav" is used by government and the media as a convenient figleaf to avoid meaningful engagement with economic and social problems on an austerity ridden island. (I'm not quite sure what the US equivalent to 'chav' might be. Perhaps Florida Man without the implications of mental health.)

Anyway, Jones certainly has a point. From Vicky Pollard to the attacks on Jade Goody, there is an ignorance and prejudice at the heart of the 'chav' backlash, and that would be bad enough even if it was not being co-opted or criminalised for ideological ends.

Elsewhere in political science, I also caught Michael Brooks' Against the Web and David Graeber's Bullshit Jobs, although they are not quite methodical enough to recommend here. However, Graeber's award-winning Debt: The First 5000 Years will be read in 2021. Matt Taibbi's Hate Inc: Why Today's Media Makes Us Despise One Another is worth a brief mention here though, but its sprawling nature felt very much like I was reading a set of Substack articles loosely edited together. And, indeed, I was.


The Golden Thread: The Story of Writing

Ewan Clayton

A recommendation from a dear friend, Ewan Clayton's The Golden Thread is a journey through the long history of the writing from the Dawn of Man to present day. Whether you are a linguist, a graphic designer, a visual artist, a typographer, an archaeologist or 'just' a reader, there is probably something in here for you. I was already dipping my quill into calligraphy this year so I suspect I would have liked this book in any case, but highlights would definitely include the changing role of writing due to the influence of textual forms in the workplace as well as digression on ergonomic desks employed by monks and scribes in the Middle Ages.

A lot of books by otherwise-sensible authors overstretch themselves when they write about computers or other technology from the Information Age, at best resulting in bizarre non-sequiturs and dangerously Panglossian viewpoints at worst. But Clayton surprised me by writing extremely cogently and accurate on the role of text in this new and unpredictable era. After finishing it I realised why — for a number of years, Clayton was a consultant for the legendary Xerox PARC where he worked in a group focusing on documents and contemporary communications whilst his colleagues were busy inventing the graphical user interface, laser printing, text editors and the computer mouse.


New Dark Age & Radical Technologies: The Design of Everyday Life

James Bridle & Adam Greenfield

I struggled to describe these two books to friends, so I doubt I will suddenly do a better job here. Allow me to quote from Will Self's review of James Bridle's New Dark Age in the Guardian:

We're accustomed to worrying about AI systems being built that will either "go rogue" and attack us, or succeed us in a bizarre evolution of, um, evolution – what we didn't reckon on is the sheer inscrutability of these manufactured minds. And minds is not a misnomer. How else should we think about the neural network Google has built so its translator can model the interrelation of all words in all languages, in a kind of three-dimensional "semantic space"?

New Dark Age also turns its attention to the weird, algorithmically-derived products offered for sale on Amazon as well as the disturbing and abusive videos that are automatically uploaded by bots to YouTube. It should, by rights, be a mess of disparate ideas and concerns, but Bridle has a flair for introducing topics which reveals he comes to computer science from another discipline altogether; indeed, on a four-part series he made for Radio 4, he's primarily referred to as "an artist".

Whilst New Dark Age has rather abstract section topics, Adam Greenfield's Radical Technologies is a rather different book altogether. Each chapter dissects one of the so-called 'radical' technologies that condition the choices available to us, asking how do they work, what challenges do they present to us and who ultimately benefits from their adoption. Greenfield takes his scalpel to smartphones, machine learning, cryptocurrencies, artificial intelligence, etc., and I don't think it would be unfair to say that starts and ends with a cynical point of view. He is no reactionary Luddite, though, and this is both informed and extremely well-explained, and it also lacks the lazy, affected and Private Eye-like cynicism of, say, Attack of the 50 Foot Blockchain.

The books aren't a natural pair, for Bridle's writing contains quite a bit of air in places, ironically mimics the very 'clouds' he inveighs against. Greenfield's book, by contrast, as little air and much lower pH value. Still, it was more than refreshing to read two technology books that do not limit themselves to platitudinal booleans, be those dangerously naive (e.g. Kevin Kelly's The Inevitable) or relentlessly nihilistic (Shoshana Zuboff's The Age of Surveillance Capitalism). Sure, they are both anti-technology screeds, but they tend to make arguments about systems of power rather than specific companies and avoid being too anti-'Big Tech' through a narrower, Silicon Valley obsessed lens — for that (dipping into some other 2020 reading of mine) I might suggest Wendy Liu's Abolish Silicon Valley or Scott Galloway's The Four.

Still, both books are superlatively written. In fact, Adam Greenfield has some of the best non-fiction writing around, both in terms of how he can explain complicated concepts (particularly the smart contract mechanism of the Ethereum cryptocurrency) as well as in the extremely finely-crafted sentences — I often felt that the writing style almost had no need to be that poetic, and I particularly enjoyed his fictional scenarios at the end of the book.


The Algebra of Happiness & Indistractable: How to Control Your Attention and Choose Your Life

Scott Galloway & Nir Eyal

A cocktail of insight, informality and abrasiveness makes NYU Professor Scott Galloway uncannily appealing to guys around my age. Although Galloway definitely has his own wisdom and experience, similar to Joe Rogan I suspect that a crucial part of Galloway's appeal is that you feel you are learning right alongside him. Thankfully, 'Prof G' is far less — err — problematic than Rogan (Galloway is more of a well-meaning, spirited centrist), although he, too, has some pretty awful takes at time. This is a shame, because removed from the whirlwind of social media he can be really quite considered, such as in this long-form interview with Stephanie Ruhle.

In fact, it is this kind of sentiment that he captured in his 2019 Algebra of Happiness. When I look over my highlighted sections, it's clear that it's rather schmaltzy out of context ("Things you hate become just inconveniences in the presence of people you love..."), but his one-two punch of cynicism and saccharine ("Ask somebody who purchased a home in 2007 if their 'American Dream' came true...") is weirdly effective, especially when he uses his own family experiences as part of his story:

A better proxy for your life isn't your first home, but your last. Where you draw your last breath is more meaningful, as it's a reflection of your success and, more important, the number of people who care about your well-being. Your first house signals the meaningful—your future and possibility. Your last home signals the profound—the people who love you. Where you die, and who is around you at the end, is a strong signal of your success or failure in life.

Nir Eyal's Indistractable, however, is a totally different kind of 'self-help' book. The important background story is that Eyal was the author of the widely-read Hooked which turned into a secular Bible of so-called 'addictive design'. (If you've ever been cornered by a techbro wielding a Wikipedia-thin knowledge of B. F. Skinner's behaviourist psychology and how it can get you to click 'Like' more often, it ultimately came from Hooked.) However, Eyal's latest effort is actually an extended mea culpa for his previous sin and he offers both high and low-level palliative advice on how to avoid falling for the tricks he so studiously espoused before. I suppose we should be thankful to capitalism for selling both cause and cure.

Speaking of markets, there appears to be a growing appetite for books in this 'anti-distraction' category, and whilst I cannot claim to have done an exhausting study of this nascent field, Indistractable argues its points well without relying on accurate-but-dry "studies show..." or, worse, Gladwellian gotchas. My main criticism, however, would be that Eyal doesn't acknowledge the limits of a self-help approach to this problem; it seems that many of the issues he outlines are an inescapable part of the alienation in modern Western society, and the only way one can really avoid distraction is to move up the income ladder or move out to a 500-acre ranch.

07 February, 2021 04:13PM

hackergotchi for Norbert Preining

Norbert Preining

New job: Fujitsu Research Labs

I am excited to announce that I have joined Fujitsu Research Labs with beginning of February.

My job will comprise, besides other things, research and development in machine learning, open source strategies, development of and representation of Fujitsu in the scikit-learn consortium. We are doing a lot of topological data analysis, so if you are interested in these kinds of topics, don’t hesitate to contact me.

I am still settling into a completely new world of “big and Japanese company” with lots of on-boarding seminars, applications, paper work, meetings, but I am looking forward to start the actual work as soon as possible.

As a long long time Linux user, I am a bit in trouble now, since everything in Fujitsu requires Windows it seems. I will try hard to improve this situation – including my dream of having Fujitsu machines with pre-installed Debian on it 😉

07 February, 2021 08:37AM by Norbert Preining