Please add Iran to AL installer

Hi

On behalf of Iranian Arch users I am very grateful for your work on AL and the great AL installer. However, I have a suggestion to improve AL installer for Iranians. We have multiple mirrors for Arch in our country but Iran is not among the country listed for choosing mirror during AL installation. Please add Iran to the list of countries if possible.

Thank you in Advance :metal:

1 Like

Hello ali, welcome to the forums :slight_smile:

While you’re waiting for a reply from the developers this should add the option from the ArchLabs live environment:

sed -i '725s/\\/IR Iran "$(ofn IR "$codes")" \\/' /usr/local/bin/installer
2 Likes

The installer should already rank your mirrors using Reflector, as there is already mirrors for Iran available then they should already be selected.

## Iran
#Server = http://mirror.bardia.tech/archlinux/$repo/os/$arch
#Server = https://mirror.bardia.tech/archlinux/$repo/os/$arch

@natemaia any ideas here?

The Iranian mirrors are available but they’re not offered as a selectable option in the installer. My posted sed command adds the option.

So sorry about this, I’m not sure how they got missed from the selection cause they’re in the code-to-country list.

I’m working on making it more dynamic and just use output from reflector --list-countries directly.

1 Like

If you hit enter with no countries selected or cancel the mirror selection dialog it will auto rank yes but it takes forever (cause it ranks a lot of mirrors) and because it has a static number of available mirrors (in this case 200) to pick from so it is possible a country with very few mirrors to be missed entirely during ranking.

The actual command we use in the installer for auto ranking is

reflector --verbose --connection-timeout 2 --threads 10 \
	--latest 200 --age 24 --score 75 --sort rate --fastest 6 --save /etc/pacman.d/mirrorlist

If you do select one or more countries from the list it will instead build a new command like

reflector --verbose --connection-timeout 2 --threads 10 "${MIRROR_COUNTRY[@]}" \
	--latest 100 --age 24 --score 50 --sort rate --fastest 6 --save /etc/pacman.d/mirrorlist

Where ${MIRROR_COUNTRY[@]} (in my selection) is an array in the form of

MIRROR_COUNTRY=(
--country "Canada"
--country "United States"
)

As stated above this will be fixed with our next installer version bump

1 Like

If you don’t want reflector in the ISO, you could use curl,

curl -s "https://archlinux.org/mirrorlist/?country=${country}&protocol=https&use_mirror_status=on" | sed -e 's/^#Server/Server/' -e '/^#/d' | rankmirrors -n 5 - > /etc/pacman.d/mirrorlist

2 Likes

We already allow using rankmirrors when reflector isn’t installed and do almost exactly like your above script, because the installer can (almost) handle being run on a stock arch iso without other packages, the one thing it still needs to work is dialog which means you’re forced to install packages regardless.

However I think it’s going unused and I should just install reflector within the installer and ditch the rankmirrors branch of script. rankmirrors is not very fast and almost no one is using it over reflector

1 Like

It depends.

~ » time reflector --verbose --connection-timeout 2 --threads 10 -c US --latest 200 --age 24 --score 75 --sort rate --fastest 6
################################################################################
################# Arch Linux mirrorlist generated by Reflector #################
################################################################################

# With:       reflector --verbose --connection-timeout 2 --threads 10 -c US --latest 200 --age 24 --score 75 --sort rate --fastest 6
# When:       2021-12-25 23:03:48 UTC
# From:       https://archlinux.org/mirrors/status/json/
# Retrieved:  2021-12-25 22:59:11 UTC
# Last Check: 2021-12-25 22:37:37 UTC

Server = http://mirrors.gigenet.com/archlinux/$repo/os/$arch
Server = http://mirror.es.its.nyu.edu/archlinux/$repo/os/$arch
Server = http://plug-mirror.rcac.purdue.edu/archlinux/$repo/os/$arch
Server = http://mirrors.advancedhosters.com/archlinux/$repo/os/$arch
Server = rsync://mirror.sfo12.us.leaseweb.net/archlinux/$repo/os/$arch
Server = http://arch.mirror.constant.com/$repo/os/$arch


________________________________________________________
Executed in   39.80 secs    fish           external
   usr time    3.83 secs  325.00 micros    3.83 secs
   sys time    4.54 secs  262.00 micros    4.54 secs
~ » time curl -s "https://archlinux.org/mirrorlist/?country=US&protocol=https&use_mirror_status=on" | sed -e 's/^#Server/Server/' -e '/^#/d' | rankmirrors -n 6 -                                                  ~
Server = https://ftp.sudhip.com/archlinux/$repo/os/$arch
Server = https://arch.mirror.constant.com/$repo/os/$arch
Server = https://mirrors.mit.edu/archlinux/$repo/os/$arch
Server = https://mirror.ette.biz/archlinux/$repo/os/$arch
Server = https://mirror.umd.edu/archlinux/$repo/os/$arch
Server = https://mirrors.rit.edu/archlinux/$repo/os/$arch

________________________________________________________
Executed in   13.29 secs    fish           external
   usr time    1.23 secs    0.29 millis    1.23 secs
   sys time    0.40 secs    1.28 millis    0.40 secs
1 Like

Absolutely, it depends on your system and thread count how fast they can execute, however

% time curl -s "https://archlinux.org/mirrorlist/?country=US&protocol=https&use_mirror_status=on" | sed -e 's/^#Server/Server/' -e '/^#/d' | rankmirrors -n 6 -
Server = https://mirrors.kernel.org/archlinux/$repo/os/$arch
Server = https://mirror.theash.xyz/arch/$repo/os/$arch
Server = https://mirrors.xtom.com/archlinux/$repo/os/$arch
Server = https://zxcvfdsa.com/arch/$repo/os/$arch
Server = https://archmirror1.octyl.net/$repo/os/$arch
Server = https://mirror.sfo12.us.leaseweb.net/archlinux/$repo/os/$arch
curl -s   0.01s user 0.00s system 1% cpu 0.708 total
sed -e 's/^#Server/Server/' -e '/^#/d'  0.00s user 0.00s system 0% cpu 0.707 total
rankmirrors -n 6 -  0.26s user 0.10s system 1% cpu 22.868 total

# say 24s overall, now lets update and install firefox

% sudo pacman -Syyu firefox
:: Synchronizing package databases...
 core                                                                       137.6 KiB   176 KiB/s 00:01 [--------------------------------------------------------------] 100%
 extra                                                                     1567.4 KiB   536 KiB/s 00:03 [--------------------------------------------------------------] 100%
 community                                                                    5.9 MiB   763 KiB/s 00:08 [--------------------------------------------------------------] 100%
 multilib                                                                   150.3 KiB   178 KiB/s 00:01 [--------------------------------------------------------------] 100%
 archlabs-testing                                                            24.0 KiB  51.9 KiB/s 00:00 [--------------------------------------------------------------] 100%
 archlabs                                                                    17.1 KiB   171 KiB/s 00:00 [--------------------------------------------------------------] 100%
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (2) mailcap-2.1.53-1  firefox-95.0.2-1

Total Download Size:    60.88 MiB
Total Installed Size:  226.54 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages...
 mailcap-2.1.53-1-any                                                        29.5 KiB  78.5 KiB/s 00:00 [--------------------------------------------------------------] 100%
 firefox-95.0.2-1-x86_64                                                     60.9 MiB  1432 KiB/s 00:44 [--------------------------------------------------------------] 100%
 Total (2/2)                                                                 60.9 MiB  1431 KiB/s 00:44 [--------------------------------------------------------------] 100%
(2/2) checking keys in keyring                                                                          [--------------------------------------------------------------] 100%
(2/2) checking package integrity                                                                        [--------------------------------------------------------------] 100%
(2/2) loading package files                                                                             [--------------------------------------------------------------] 100%
(2/2) checking for file conflicts                                                                       [--------------------------------------------------------------] 100%
:: Processing package changes...
(1/2) installing mailcap                                                                                [--------------------------------------------------------------] 100%
(2/2) installing firefox                                                                                [--------------------------------------------------------------] 100%
Optional dependencies for firefox
    networkmanager: Location detection via available WiFi networks
    libnotify: Notification integration [installed]
    pulseaudio: Audio support [installed]
    speech-dispatcher: Text-to-Speech
    hunspell-en_US: Spell checking, American English
    xdg-desktop-portal: Screensharing with Wayland
:: Running post-transaction hooks...
(1/3) Arming ConditionNeedsUpdate...
(2/3) Updating icon theme caches...
(3/3) Updating the desktop file MIME type cache...

Hmm, that’s not great (44s to install firefox) let’s try reflector

% time reflector --verbose --connection-timeout 2 --threads 10 -c US --latest 200 --age 24 --score 75 --sort rate --fastest 6
Server = https://zxcvfdsa.com/arch/$repo/os/$arch
Server = https://archmirror1.octyl.net/$repo/os/$arch
Server = https://mirrors.xtom.com/archlinux/$repo/os/$arch
Server = https://mirror.sfo12.us.leaseweb.net/archlinux/$repo/os/$arch
Server = https://mirrors.radwebhosting.com/archlinux/$repo/os/$arch
Server = http://mirror.sfo12.us.leaseweb.net/archlinux/$repo/os/$arch

reflector --verbose --connection-timeout 2 --threads 10 -c US --latest 200  2  0.49s user 0.36s system 3% cpu 26.485 total

# say 27s to be fair
# alright reflector is slower to rank but let's update and install firefox


% sudo pacman -Syyu firefox
:: Synchronizing package databases...
 core                                                                       137.5 KiB   426 KiB/s 00:00 [--------------------------------------------------------------] 100%
 extra                                                                     1568.3 KiB  3.19 MiB/s 00:00 [--------------------------------------------------------------] 100%
 community                                                                    5.9 MiB  10.3 MiB/s 00:01 [--------------------------------------------------------------] 100%
 multilib                                                                   150.3 KiB   479 KiB/s 00:00 [--------------------------------------------------------------] 100%
 archlabs-testing                                                            24.0 KiB  48.9 KiB/s 00:00 [--------------------------------------------------------------] 100%
 archlabs                                                                    17.1 KiB  65.2 KiB/s 00:00 [--------------------------------------------------------------] 100%
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (2) mailcap-2.1.53-1  firefox-95.0.2-1

Total Download Size:    60.88 MiB
Total Installed Size:  226.54 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages...
 mailcap-2.1.53-1-any                                                        29.5 KiB   214 KiB/s 00:00 [--------------------------------------------------------------] 100%
 firefox-95.0.2-1-x86_64                                                     60.9 MiB  34.3 MiB/s 00:02 [--------------------------------------------------------------] 100%
 Total (2/2)                                                                 60.9 MiB  33.4 MiB/s 00:02 [--------------------------------------------------------------] 100%
(2/2) checking keys in keyring                                                                          [--------------------------------------------------------------] 100%
(2/2) checking package integrity                                                                        [--------------------------------------------------------------] 100%
(2/2) loading package files                                                                             [--------------------------------------------------------------] 100%
(2/2) checking for file conflicts                                                                       [--------------------------------------------------------------] 100%
:: Processing package changes...
(1/2) installing mailcap                                                                                [--------------------------------------------------------------] 100%
(2/2) installing firefox                                                                                [--------------------------------------------------------------] 100%
Optional dependencies for firefox
    networkmanager: Location detection via available WiFi networks
    libnotify: Notification integration [installed]
    pulseaudio: Audio support [installed]
    speech-dispatcher: Text-to-Speech
    hunspell-en_US: Spell checking, American English
    xdg-desktop-portal: Screensharing with Wayland
:: Running post-transaction hooks...
(1/3) Arming ConditionNeedsUpdate...
(2/3) Updating icon theme caches...
(3/3) Updating the desktop file MIME type cache...

As you can see I installed firefox in about 2s, that’s a 20x speed improvement over the mirrors from rankmirrors (44s), checking for updates is also faster (2s vs 14s). These things add up during install when it’s a lot more than just firefox being downloaded and installed. In a pinch rankmirrors is a good program and it does return usable mirrors, they’re just more often than not unbearably slow. It’s very hard to test these things locally as the servers could be far/close depending on the test location but overall I’ve received numerous bumps from people about how fast our server ranking makes package installs. reflector can also be made to go much much faster (on my system) by giving it more threads but this comes at the cost of accuracy (if the help page is to be believed)

I don’t want to sound like an ass, I love conversations like this where I need to rethink things and make sure regardless of whether I’m right or wrong so thank you :slight_smile: genuinely.

2 Likes

The mirrorlist you get could be dynamic, which means you might have two different lists when you run the command twice. So, I care more about how long it takes to get/sort the server list. As for how long it takes to install a package, it depends on a lot of things, your connection speed, dependency pkgs, your file systems, etc. I don’t know how it took 44s to install firefox in your system. I am using a much older/slower machine. Here is my time. I am on Obarun though, not sure if the systemd plays any role in the test.

~ » doas pacman -Rns firefox                                                                                                                                                         ~
checking dependencies...

Packages (2) mailcap-2.1.53-1  firefox-95.0.2-1

Total Removed Size:  226.54 MiB

:: Do you want to remove these packages? [Y/n] 
:: Processing package changes...
(1/2) removing firefox                                                                                        [#################################################################] 100%
(2/2) removing mailcap                                                                                        [#################################################################] 100%
:: Running post-transaction hooks...
(1/2) Updating icon theme caches...
(2/2) Updating the desktop file MIME type cache...

~ » doas pacman -Scc                                                                                                                                                                 ~

Cache directory: /var/cache/pacman/pkg/
:: Do you want to remove ALL files from cache? [y/N] y
removing all files from cache...

Database directory: /var/lib/pacman/
:: Do you want to remove unused repositories? [Y/n] y
removing unused sync repositories...

~ » time doas pacman -Syyu firefox                                                                                                                                                   ~
:: Synchronizing package databases...
 obcore                                                                            28.4 KiB  44.0 KiB/s 00:01 [#################################################################] 100%
 obextra                                                                           99.3 KiB   141 KiB/s 00:01 [#################################################################] 100%
 obcommunity                                                                       23.1 KiB  38.3 KiB/s 00:01 [#################################################################] 100%
 observice                                                                         26.8 KiB  45.0 KiB/s 00:01 [#################################################################] 100%
 core                                                                             137.6 KiB  1349 KiB/s 00:00 [#################################################################] 100%
 extra                                                                           1567.4 KiB  7.32 MiB/s 00:00 [#################################################################] 100%
 community                                                                          5.9 MiB  6.87 MiB/s 00:01 [#################################################################] 100%
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (2) mailcap-2.1.53-1  firefox-95.0.2-1

Total Download Size:    60.88 MiB
Total Installed Size:  226.54 MiB

:: Proceed with installation? [Y/n] 
:: Retrieving packages...
 mailcap-2.1.53-1-any                                                              29.5 KiB   410 KiB/s 00:00 [#################################################################] 100%
 firefox-95.0.2-1-x86_64                                                           60.9 MiB  8.97 MiB/s 00:07 [#################################################################] 100%
 Total (2/2)                                                                       60.9 MiB  8.97 MiB/s 00:07 [#################################################################] 100%
(2/2) checking keys in keyring                                                                                [#################################################################] 100%
(2/2) checking package integrity                                                                              [#################################################################] 100%
(2/2) loading package files                                                                                   [#################################################################] 100%
(2/2) checking for file conflicts                                                                             [#################################################################] 100%
(2/2) checking available disk space                                                                           [#################################################################] 100%
:: Processing package changes...
(1/2) installing mailcap                                                                                      [#################################################################] 100%
(2/2) installing firefox                                                                                      [#################################################################] 100%
Optional dependencies for firefox
    networkmanager: Location detection via available WiFi networks [installed]
    libnotify: Notification integration [installed]
    pulseaudio: Audio support
    speech-dispatcher: Text-to-Speech
    hunspell-en_US: Spell checking, American English
    xdg-desktop-portal: Screensharing with Wayland
:: Running post-transaction hooks...
(1/2) Updating icon theme caches...
(2/2) Updating the desktop file MIME type cache...

________________________________________________________
Executed in   13.46 secs    fish           external
   usr time    3.61 secs    0.00 micros    3.61 secs
   sys time    3.02 secs  800.00 micros    3.02 secs

My mirrorlist,

Server = http://arch.mirror.constant.com/$repo/os/$arch
Server = http://mirrors.mit.edu/archlinux/$repo/os/$arch
Server = http://mirror.ette.biz/archlinux/$repo/os/$arch
Server = http://iad.mirrors.misaka.one/archlinux/$repo/os/$arch
Server = http://mirror.siena.edu/archlinux/$repo/os/$arch
Server = http://mirror.clarkson.edu/archlinux/$repo/os/$arch
Server = http://mirrors.kernel.org/archlinux/$repo/os/$arch
Server = http://mirrors.rit.edu/archlinux/$repo/os/$arch
Server = http://mirror.pit.teraswitch.com/archlinux/$repo/os/$arch
Server = http://mirrors.gigenet.com/archlinux/$repo/os/$arch

Did you remember to clean your cache beforehand?

The 44s could be an outlier but to be fair I ran it 3 times and took the best result, There’s a lot at play here so like I said it’s hard to test

You mean this?

~ » doas pacman -Scc

My mistake I missed that :smiley:

You have encryption on your file system? It sounds a little odd it took that long.

No worries, mate. It’s fun to talk about things like that.

1 Like

No just a bog standard ext4 install, same one I’ve been running for years

Did you try running it with both mirrorlists to see the difference? The 44s is not really important, what is important is the comparison between the two - apples to apples - sort of.

Should be the same. I had “http://arch.mirror.constant.com/$repo/os/$arch” as the top server in the list.

Thank you very much Sir. I will wait to see how things goes after the upgrading the installer.