zoneinfo.available_timezones() sometimes include 'localtime'

[hongquan]

Bug report

Bug description:

Sometimes available_timezones includes localtime. This result is not stable. But localtime should not be included at all.

>>> import zones

>>> zones = zoneinfo.available_timezones()
>>> 'localtime' in zones
True

Information about the data folder where this function scans:

❯ ll /usr/share/zoneinfo
Permissions Size User Group Date Modified Name
drwxr-xr-x     - root root   3 Jul 20:37  Africa
drwxr-xr-x     - root root   3 Jul 20:37  America
drwxr-xr-x     - root root   3 Jul 20:37  Antarctica
drwxr-xr-x     - root root   3 Jul 20:37  Arctic
drwxr-xr-x     - root root   3 Jul 20:37  Asia
drwxr-xr-x     - root root   3 Jul 20:37  Atlantic
drwxr-xr-x     - root root   3 Jul 20:37  Australia
drwxr-xr-x     - root root   3 Jul 20:37  Etc
drwxr-xr-x     - root root   3 Jul 20:37  Europe
drwxr-xr-x     - root root   3 Jul 20:37  Indian
drwxr-xr-x     - root root   3 Jul 20:37  Pacific
.rw-r--r--   116 root root  22 Apr 16:56  Factory
lrwxrwxrwx     - root root  22 Apr 16:56  GMT -> Etc/GMT
.rw-r--r--  4.8k root root   6 Sep  2023  iso3166.tab
.rw-r--r--  5.1k root root   7 Jan 18:46  leap-seconds.list
.rw-r--r--  3.3k root root  22 Apr 16:56  leapseconds
lrwxrwxrwx     - root root  22 Apr 16:56  localtime -> /etc/localtime
lrwxrwxrwx     - root root  22 Apr 16:56  posixrules -> America/New_York
.rw-r--r--  114k root root  22 Apr 16:56  tzdata.zi
lrwxrwxrwx     - root root  22 Apr 16:56  UTC -> Etc/UTC
.rw-r--r--   19k root root  20 Mar 18:43  zone.tab
.rw-r--r--   18k root root  20 Mar 18:43  zone1970.tab
.rw-r--r--  8.1k root root  22 Mar 20:21  zonenow.tab

/etc/localtime again, is a symlink:

❯ ll /etc/localtime
Permissions Size User Group Date Modified Name
lrwxrwxrwx     - root root   8 Jan 12:15  /etc/localtime -> /usr/share/zoneinfo/Asia/Ho_Chi_Minh

I think in this block, the file path should be resolved for symlink:

cpython/Lib/zoneinfo/_tzpath.py

Lines 157 to 167 in d22a745

	fpath = os.path.join(root, file)
	key = os.path.relpath(fpath, start=tz_root)
	if os.sep != "/": # pragma: nocover
	key = key.replace(os.sep, "/")
	if not key or key in valid_zones:
	continue
	if valid_key(fpath):
	valid_zones.add(key)

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

• gh-137976: Exclude symlinks from zoneinfo.available_timezones() results #137977
• gh-137976: fix available_timezones to prevent reporting an invalid IANA zone name: localtime #138012

[hongquan]

I think it is not correct to just ignore the symlinks. Sometimes, two cities have the same rules, and the data author use symlinks to point these timezones to the same source.

❯ ll /usr/share/zoneinfo/Asia/
Permissions Size User Group Date Modified Name
.rw-r--r--   151 root root  22 Apr 16:56  Aden
.rw-r--r--   983 root root  22 Apr 16:56  Almaty
.rw-r--r--  1.4k root root  22 Apr 16:56  Amman
.rw-r--r--  1.2k root root  22 Apr 16:56  Anadyr
.rw-r--r--   969 root root  22 Apr 16:56  Aqtau
.rw-r--r--   997 root root  22 Apr 16:56  Aqtobe
.rw-r--r--   605 root root  22 Apr 16:56  Ashgabat
.rw-r--r--   977 root root  22 Apr 16:56  Atyrau
.rw-r--r--   969 root root  22 Apr 16:56  Baghdad
.rw-r--r--   223 root root  22 Apr 16:56  Bahrain
.rw-r--r--  1.2k root root  22 Apr 16:56  Baku
.rw-r--r--   185 root root  22 Apr 16:56  Bangkok
.rw-r--r--  1.2k root root  22 Apr 16:56  Barnaul
.rw-r--r--  2.2k root root  22 Apr 16:56  Beirut
.rw-r--r--   969 root root  22 Apr 16:56  Bishkek
.rw-r--r--   189 root root  22 Apr 16:56  Brunei
.rw-r--r--  1.2k root root  22 Apr 16:56  Chita
lrwxrwxrwx     - root root  22 Apr 16:56  Chongqing -> Shanghai
.rw-r--r--   358 root root  22 Apr 16:56  Colombo
.rw-r--r--  1.9k root root  22 Apr 16:56  Damascus
.rw-r--r--   323 root root  22 Apr 16:56  Dhaka
.rw-r--r--   257 root root  22 Apr 16:56  Dili
.rw-r--r--   151 root root  22 Apr 16:56  Dubai
.rw-r--r--   577 root root  22 Apr 16:56  Dushanbe
.rw-r--r--  2.0k root root  22 Apr 16:56  Famagusta
.rw-r--r--  3.8k root root  22 Apr 16:56  Gaza
lrwxrwxrwx     - root root  22 Apr 16:56  Harbin -> Shanghai
.rw-r--r--  3.9k root root  22 Apr 16:56  Hebron
.rw-r--r--   337 root root  22 Apr 16:56  Ho_Chi_Minh
.rw-r--r--  1.2k root root  22 Apr 16:56  Hong_Kong
.rw-r--r--   877 root root  22 Apr 16:56  Hovd
.rw-r--r--  1.2k root root  22 Apr 16:56  Irkutsk
lrwxrwxrwx     - root root  22 Apr 16:56  Istanbul -> ../Europe/Istanbul
.rw-r--r--   383 root root  22 Apr 16:56  Jakarta
.rw-r--r--   221 root root  22 Apr 16:56  Jayapura
.rw-r--r--  2.4k root root  22 Apr 16:56  Jerusalem
.rw-r--r--   194 root root  22 Apr 16:56  Kabul
.rw-r--r--  1.2k root root  22 Apr 16:56  Kamchatka
.rw-r--r--   379 root root  22 Apr 16:56  Karachi
lrwxrwxrwx     - root root  22 Apr 16:56  Kashgar -> Urumqi
.rw-r--r--   198 root root  22 Apr 16:56  Kathmandu
.rw-r--r--  1.3k root root  22 Apr 16:56  Khandyga
.rw-r--r--   285 root root  22 Apr 16:56  Kolkata
.rw-r--r--  1.2k root root  22 Apr 16:56  Krasnoyarsk
.rw-r--r--   401 root root  22 Apr 16:56  Kuala_Lumpur
.rw-r--r--   469 root root  22 Apr 16:56  Kuching
.rw-r--r--   151 root root  22 Apr 16:56  Kuwait
.rw-r--r--  1.2k root root  22 Apr 16:56  Macau
.rw-r--r--  1.2k root root  22 Apr 16:56  Magadan
.rw-r--r--   254 root root  22 Apr 16:56  Makassar
.rw-r--r--   422 root root  22 Apr 16:56  Manila
.rw-r--r--   151 root root  22 Apr 16:56  Muscat
.rw-r--r--  2.0k root root  22 Apr 16:56  Nicosia
.rw-r--r--  1.2k root root  22 Apr 16:56  Novokuznetsk
.rw-r--r--  1.2k root root  22 Apr 16:56  Novosibirsk
.rw-r--r--  1.2k root root  22 Apr 16:56  Omsk
.rw-r--r--   991 root root  22 Apr 16:56  Oral
.rw-r--r--   281 root root  22 Apr 16:56  Phnom_Penh
.rw-r--r--   353 root root  22 Apr 16:56  Pontianak
.rw-r--r--   237 root root  22 Apr 16:56  Pyongyang
.rw-r--r--   185 root root  22 Apr 16:56  Qatar
.rw-r--r--  1.0k root root  22 Apr 16:56  Qostanay
.rw-r--r--  1.0k root root  22 Apr 16:56  Qyzylorda
.rw-r--r--   151 root root  22 Apr 16:56  Riyadh
.rw-r--r--  1.2k root root  22 Apr 16:56  Sakhalin
.rw-r--r--   563 root root  22 Apr 16:56  Samarkand
.rw-r--r--   617 root root  22 Apr 16:56  Seoul
.rw-r--r--   561 root root  22 Apr 16:56  Shanghai
.rw-r--r--   401 root root  22 Apr 16:56  Singapore
.rw-r--r--  1.2k root root  22 Apr 16:56  Srednekolymsk
.rw-r--r--   761 root root  22 Apr 16:56  Taipei
.rw-r--r--   577 root root  22 Apr 16:56  Tashkent
.rw-r--r--  1.0k root root  22 Apr 16:56  Tbilisi
.rw-r--r--  1.2k root root  22 Apr 16:56  Tehran
lrwxrwxrwx     - root root  22 Apr 16:56  Tel_Aviv -> Jerusalem
.rw-r--r--   189 root root  22 Apr 16:56  Thimphu
.rw-r--r--   309 root root  22 Apr 16:56  Tokyo
.rw-r--r--  1.2k root root  22 Apr 16:56  Tomsk
.rw-r--r--   877 root root  22 Apr 16:56  Ulaanbaatar
.rw-r--r--   151 root root  22 Apr 16:56  Urumqi
.rw-r--r--  1.2k root root  22 Apr 16:56  Ust-Nera
.rw-r--r--   309 root root  22 Apr 16:56  Vientiane
.rw-r--r--  1.2k root root  22 Apr 16:56  Vladivostok
.rw-r--r--  1.2k root root  22 Apr 16:56  Yakutsk
.rw-r--r--   254 root root  22 Apr 16:56  Yangon
.rw-r--r--  1.2k root root  22 Apr 16:56  Yekaterinburg
.rw-r--r--  1.1k root root  22 Apr 16:56  Yerevan

[serhiy-storchaka]

Concur with @hongquan. Note also that "GMT" is a symlink -- you don't want to get rid of it, do you? And there is a timezone with strange name "Factory".

[hongquan]

I think, to decide which files to take, we can require the folder + file name to follow the scheme "Continent/City". We can ignore those like "localtime", "Etc/xxx", "GMT".

[ntindle]

I also experienced this issue today on GitHub Actions runners. The list available on the runners differs from the timezone list pulled on my Mac by only localtime, which is not an IANA timezone and thus should not be returned based on the mission of PEP 615

[hongquan]

I think the solution should be future-proof. The Linux distro maintainers may add more file similar to localtime in the future, so excluding only this file like PR #138012 is not enough.

Rectification for my previous comment: "Etc/GMT-xxx" should be taken. By https://data.iana.org/time-zones/tzdb/etcetera, they are "for ships at sea and other miscellany".