TrueNAS Apps catalog fails to sync - Git HTTP/2 returns 401, HTTP/1.1 works

Apps and Virtualization
, ,
21

It has worked for me now. I first tried to clone the repo on both my EndeavourOS box and a Windows box and both worked without authentication, so I decided to try again on my TrueNAS box and it worked as well. I didn’t make any changes to how git behaves on my TrueNAS box.

It might have something to do with them throttling/blocking non-authenticated accounts when there is too much load or something. I work at a university in a building which has a lot of PhDs and Postdocs and there are times when I can’t even properly access github repos via the website. I don’t remember the specific error it gives, but it basically says that there are too many requests for our IP (I can not imagine how many of those they get for the full university).

So it is possible they block non-authenticated accounts at some points when the load is too high.

22

git config --system http.version HTTP/1.1 fixed it for me too. I was suggested to undo this after an update of Truenas will be okay again updating the catalog by running git config --system --unset http.version.

23

Hmm, so is the recommendation to run the workaround or what should we do?

24

well, here everthing’s fine now.. github fixed its issue at least for me

25

Good catch!

I was having the same issue.
Strangly enough when running

GIT_CURL_VERBOSE=1 git clone https://github.com/truenas/apps/tmp/apps-test 2>&1 | grep -i "HTTP/"

The HTTP version return is http2 and the sync went okay. Maybe the problem resides in the middleware layer. This is what I saw:

  • GUI

When running a manual apps sync from the gui, it deletes the content of the truenas_catalog directory. But does not start syncing. It fails with

[EFAULT] Failed to clone 'https://github.com/truenas/apps' repository at '/mnt/.ix-apps/truenas_catalog' destination: [EFAULT] Failed to clone 'https://github.com/truenas/apps' repository at '/mnt/.ix-apps/truenas_catalog' destination: Cloning into '/mnt/.ix-apps/truenas_catalog'......

  • CLI

When running the sync manually from the shell the sync runs okay midclt call catalog.sync, occuping the truenas_catalog directory with data

As I said above running the git clone command in verbose mode reports back that the HTTP version used is HTTP/2. This would mean that git clone is not problem when it works with HTTP/2.

This only (i think) leaves the middleware stack between the CLI and GUI.
After I forced the git client to http/1.1 the GUI sync went okay.

26

Follow-up this is indeed a GitHub issue :slight_smile: Unauthenticated clone of a public repo intermittently returns 401 (prompts for Username) from CI workers · community · Discussion #206581 · GitHub

27

Well it works now, i guess it got fixed by Github?

28

Heh….. Unauthenticated clone of a public repo intermittently returns 401 (prompts for Username) from CI workers · community · Discussion #206581 · GitHub

EDIT: The implications of their changes are being considered on our side. I’ve got no immediate solution since we’ll need to do some research on our side. Will update the thread as necessary.

29

Thanks!

Therefore, if possible, we recommend configuring your Git client with authentication credentials for GitHub. If you need assistance with this, we have documented options in “Authenticating with the command line.”

I just find it crazy they decided randomly without any announcements or anything to slowly kill unauthenticated pulls lol. Yes it is “easy” to fix by just configuring an authentication for github in TrueNAS (or in this case any application that was doing unauth pulls).

I am just disappointed it went this route and not the official way lol

30

I have just started to use TrueNAS for my home server but cannot get the Catalog to sync. I have tried the suggested git config --system http.version HTTP/1.1 does not work, well not for me anyway. Everyone here talks like everyone else knows what they are talking about. You yourself say that, “Yes it is “easy” to fix by just configuring an authentication for github in TrueNAS” but you don’t say HOW to do it for us noobs who need to know what to do. Please can you, someone, explain for the noobs what we have to do to get this catalog to sync PLEASE?

31

Right now there is no option to configure auth for GitHub, and I did not apply the change and my catalogue started to work again after 2 days… You may have another issue and should probably start your own thread to diagnose…

32

Seeing a very similar error, but different local path:

Failed to sync TRUENAS catalog: [EFAULT] Failed to clone 'https:// github . com /truenas/apps' repository at '/var/run/middleware/ix-apps/catalogs' destination: [EFAULT] Failed to clone 'https:// github . com / truenas/apps' repository at '/var/run/middleware/ix-apps/catalogs' destination: Cloning into '/var/run/middleware/ix-...

I’m on SCALE 25.10.3 .

However, I keep my TrueNAS disconnected from the internet, except for necessary updates. I also don’t use any Apps, dockets or containers.

  1. Is this the same issue?
  2. Is this specifically for apps, separate from the mechanism that checks for OS updates?
  3. Any way to disable these automatic attempts to update the apps catalog? If that’s what it is. Or should I ignore these errors?
33

That’s rather old at this point.

No, the issue you’re seeing is that it can’t check the apps repository because you aren’t connected to the Internet.

I’d expect an error in checking for OS updates, but that would be distinct from this one.

Pretty much this.