Pretranslation breaks for accesskey labels with variant keys of type NumberLiteral

[flodolo]

@andy-moz mentioned that pretranslation didn't run automatically for Swedish in a few occasions (notably, new strings for Firefox and Firefox for Android this week).

Firefox had 6 translations missing, and manually pretranslation worked for them. Trying to see if I can spot anything in the logs.

Pretranslation worked as expected for me, earlier this week (Italian, Mozilla accounts).

[flodolo]

This might be the relevant part of the log (Firefox sync happened at 8:02 on Nov 28)

Fetching pretranslations for locale sv-SE started
[2024-11-28 08:10:37,390: ERROR/ForkPoolWorker-97] Task pontoon.sync.tasks.sync_project[5c6817ab-283b-4ccf-a254-7407ce3a5622] raised unexpected: AttributeError("'NumberLiteral' object has no attribute 'name'")
Traceback (most recent call last):
  File "/app/.heroku/python/lib/python3.11/site-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/newrelic/hooks/application_celery.py", line 100, in wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/celery/app/trace.py", line 736, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/sync/core.py", line 71, in wrapped_func
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/sync/tasks.py", line 106, in sync_project
    sync_translations(
  File "/app/pontoon/sync/tasks.py", line 407, in sync_translations
    pretranslate(db_project.pk, locales=locales, entities=entities)
  File "/app/.heroku/python/lib/python3.11/site-packages/celery/local.py", line 182, in __call__
    return self._get_current_object()(*a, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/celery/app/trace.py", line 737, in __protected_call__
    return orig(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/newrelic/hooks/application_celery.py", line 96, in wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/celery/app/task.py", line 411, in __call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/sync/core.py", line 71, in wrapped_func
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/pretranslation/tasks.py", line 125, in pretranslate
    pretranslations = get_pretranslations(entity, locale)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/pretranslation/pretranslate.py", line 56, in get_pretranslations
    pretranslate.visit(entry)
  File "/app/.heroku/python/lib/python3.11/site-packages/fluent/syntax/visitor.py", line 47, in visit
    return visit(node)
           ^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/fluent/syntax/visitor.py", line 54, in generic_visit
    new_val = self.visit(child)
              ^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/fluent/syntax/visitor.py", line 47, in visit
    return visit(node)
           ^^^^^^^^^^^
  File "/app/pontoon/pretranslation/transformer.py", line 205, in visit_Attribute
    if set_accesskey(element):
       ^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/pretranslation/transformer.py", line 194, in set_accesskey
    candidates = extract_accesskey_candidates(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/pretranslation/transformer.py", line 129, in extract_accesskey_candidates
    source = get_source(["label", "value", "aria-label"])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/pretranslation/transformer.py", line 109, in get_source
    variant = next(
              ^^^^^
  File "/app/pontoon/pretranslation/transformer.py", line 110, in <genexpr>
    (v for v in variants if v.key.name == variant_name), variants[0]
                            ^^^^^^^^^^
AttributeError: 'NumberLiteral' object has no attribute 'name'

[flodolo]

For Android, I think the problem is an integrity_error

Fetching pretranslations for locale sv-SE started
[2024-11-27 16:10:42,842: ERROR/ForkPoolWorker-3] Task pontoon.sync.tasks.sync_project[15f776c0-25d4-4423-9ea8-60d3c61ac29d] raised unexpected: IntegrityError('duplicate key value violates unique constraint "entity_locale_active"\\nDETAIL:  Key (entity_id, locale_id, active)=(313530, 234, t) already exists.\\n')
Traceback (most recent call last):
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/newrelic/hooks/database_psycopg2.py", line 61, in execute
    return super(CursorWrapper, self).execute(sql, parameters, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/newrelic/hooks/database_dbapi2.py", line 37, in execute
    return self.__wrapped__.execute(sql, parameters,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "entity_locale_active"
DETAIL:  Key (entity_id, locale_id, active)=(313530, 234, t) already exists.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/app/.heroku/python/lib/python3.11/site-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/newrelic/hooks/application_celery.py", line 100, in wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/celery/app/trace.py", line 736, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/sync/core.py", line 71, in wrapped_func
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/sync/tasks.py", line 106, in sync_project
    sync_translations(
  File "/app/pontoon/sync/tasks.py", line 407, in sync_translations
    pretranslate(db_project.pk, locales=locales, entities=entities)
  File "/app/.heroku/python/lib/python3.11/site-packages/celery/local.py", line 182, in __call__
    return self._get_current_object()(*a, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/celery/app/trace.py", line 737, in __protected_call__
    return orig(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/newrelic/hooks/application_celery.py", line 96, in wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/celery/app/task.py", line 411, in __call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/sync/core.py", line 71, in wrapped_func
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/pretranslation/tasks.py", line 175, in pretranslate
    translations = Translation.objects.bulk_create(translations)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/query.py", line 803, in bulk_create
    returned_columns = self._batched_insert(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/query.py", line 1831, in _batched_insert
    self._insert(
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/query.py", line 1805, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
    cursor.execute(sql, params)
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/app/.heroku/python/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/newrelic/hooks/database_psycopg2.py", line 61, in execute
    return super(CursorWrapper, self).execute(sql, parameters, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/newrelic/hooks/database_dbapi2.py", line 37, in execute
    return self.__wrapped__.execute(sql, parameters,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.IntegrityError: duplicate key value violates unique constraint "entity_locale_active"
DETAIL:  Key (entity_id, locale_id, active)=(313530, 234, t) already exists.

[mathjazz]

This might be the relevant part of the log (Firefox sync happened at 8:02 on Nov 28)

Found the Raygun exception in my... Spam.

This is the string in question:
https://pontoon.mozilla.org/sv-SE/firefox/all-resources/?string=313552

The error is reproducible. See the snippet below.

The code that extracts access key candidates expects that if the source attribute is a SelectExpression, its variant keys must have a name property. That's not the case for variant keys of type NumberLiteral.

from pontoon.base.models import *
from pontoon.pretranslation.pretranslate import get_pretranslations

entity = Entity.objects.get(pk=313552)
locale = Locale.objects.get(code="sv-SE")

get_pretranslations(entity, locale, False)

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/app/pontoon/pretranslation/pretranslate.py", line 56, in get_pretranslations
    pretranslate.visit(entry)
  File "/app/.heroku/python/lib/python3.11/site-packages/fluent/syntax/visitor.py", line 47, in visit
    return visit(node)
           ^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/fluent/syntax/visitor.py", line 54, in generic_visit
    new_val = self.visit(child)
              ^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.11/site-packages/fluent/syntax/visitor.py", line 47, in visit
    return visit(node)
           ^^^^^^^^^^^
  File "/app/pontoon/pretranslation/transformer.py", line 205, in visit_Attribute
    if set_accesskey(element):
       ^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/pretranslation/transformer.py", line 194, in set_accesskey
    candidates = extract_accesskey_candidates(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/pretranslation/transformer.py", line 129, in extract_accesskey_candidates
    source = get_source(["label", "value", "aria-label"])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/pontoon/pretranslation/transformer.py", line 109, in get_source
    variant = next(
              ^^^^^
  File "/app/pontoon/pretranslation/transformer.py", line 110, in <genexpr>
    (v for v in variants if v.key.name == variant_name), variants[0]
                            ^^^^^^^^^^
AttributeError: 'NumberLiteral' object has no attribute 'name'